Fix job statistics bugs and improve docs

[bgunebakan]

Summary of the changes / Why this is an improvement

Investigating #871 showed that ctk info jobs and ctk cfr jobstats are not two implementations of the same thing, so this PR does not consolidate them.

ctk info jobs takes a one-shot snapshot and lets CrateDB aggregate it in SQL, while ctk cfr jobstats polls continuously and accumulates call counters, duration histograms, and a decaying average in Python, so statistics outlive the bounded sys.jobs_log.

it is very difficult to understand which command does what from the documentation"*. So this PR keeps both engines, documents what each one is for, and fixes the bugs found while reading them.

Bug fixes

• collect --anonymize <file> was declared is_flag=True, so passing the decoder dictionary path the help text advertised failed outright.

• collect --anonymize fell back to the original statement when anonymization raised, storing in clear text exactly what the option exists to protect. Such statements are now redacted to a stable digest, so the option fails closed.

• view --deanonymize emitted unparseable output.

• view --deanonymize reported every statement twice.

• view mixed up fields, reporting the average duration as the duration histogram, the histogram as the user, and the user as the query type.

• collect counted a job twice when it ended exactly on the watermark millisecond.

• collect derived its watermark from an arbitrary record whenever the watermark table held more than one.

• collect never resumed where a previous run left off. the watermark was written but never read back.

• collect lost statistics when started twice within one process.

• report and ui refused to run whenever statistics had been collected into a schema other than stats.

• top100_count aliased PERCENTILE(..., 0.99) as p90. The value was always the 99th percentile, only the label was wrong. The field is named p99 now, which breaks consumers reading p90.

Checklist

• Link to issue this PR refers to (if applicable): Fixes Consolidate the two job-statistics implementations (ctk info jobs vs ctk cfr jobstats) #871
• Add changes in CHANGES.md

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a4b8f2d6-3e97-4bf0-8c0b-3671b28f1487

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}