-
Notifications
You must be signed in to change notification settings - Fork 854
docs: Guide on how to setup UTF-8 mode in Cortex #7439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
RaphSku
wants to merge
1
commit into
cortexproject:master
Choose a base branch
from
RaphSku:docs/utf8-support-guide
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,75 @@ | ||||||
| --- | ||||||
| title: "UTF-8 support" | ||||||
| linkTitle: "UTF-8 support" | ||||||
| weight: 70 | ||||||
| slug: utf8-support | ||||||
| --- | ||||||
|
|
||||||
| # UTF-8 support | ||||||
|
|
||||||
| ## Why UTF-8 support? | ||||||
|
|
||||||
| There are a number of reasons why UTF-8 support in Cortex is useful: | ||||||
|
|
||||||
| 1. With Prometheus version [3.0](https://prometheus.io/docs/guides/utf8/) UTF-8 support has been released for metric and label names. Thus, UTF-8 support improves the interoperability between Cortex and modern Prometheus ecosystems. | ||||||
| 2. It makes Cortex more flexible for internationalized or externally sourced metadata which is useful when metrics or labels originate from systems, teams, or domains that do not fit neatly into ASCII-only naming. | ||||||
| 3. It keeps behavior consistent across ingestion, rules and alerting. | ||||||
|
|
||||||
| ## Before you enable UTF-8 support | ||||||
|
|
||||||
| Although UTF-8 support increases flexibility, the traditional naming style remains the safest choice for broad ecosystem compatibility, because downstream tools might assume legacy-compatible names. Definitely, check this before enabling UTF-8 support. | ||||||
|
|
||||||
| Before enabling UTF-8 support, review the systems that write metrics to Cortex, the rules evaluated by the Ruler, and the alerting configuration used by Alertmanager. | ||||||
|
|
||||||
| Enabling UTF-8 support changes validation behavior. Roll it out in a staging environment first and verify that ingestion, rule evaluation, and alert routing continue to work as expected. | ||||||
|
|
||||||
| ## Configure UTF-8 support | ||||||
|
|
||||||
| Cortex supports configuring how metric names and label names are validated through the `-name-validation-scheme` flag or `name_validation_scheme` in the YAML configuration file. | ||||||
|
|
||||||
| Supported values are: | ||||||
|
|
||||||
| - `legacy` (default) | ||||||
| - `utf8` | ||||||
|
|
||||||
| The UTF-8 support has been released since Cortex [v1.20.0](https://github.com/cortexproject/cortex/releases/tag/v1.20.0). | ||||||
|
|
||||||
| This guide explains how to enable UTF-8 support and how the selected validation scheme affects the Cortex components. | ||||||
|
|
||||||
| To enable UTF-8 validation, set the validation scheme to `utf8`. | ||||||
|
|
||||||
| CLI: | ||||||
|
|
||||||
| ```bash | ||||||
| -name-validation-scheme=utf8 | ||||||
| ``` | ||||||
|
|
||||||
| YAML: | ||||||
|
|
||||||
| ```yaml | ||||||
| name_validation_scheme: utf8 | ||||||
| ``` | ||||||
|
|
||||||
| ### Impact on Cortex components | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since the Distributor is a sub-item, it would be better to increase to the
Suggested change
|
||||||
|
|
||||||
| ### Distributor | ||||||
|
|
||||||
| The [Distributor](https://cortexmetrics.io/docs/architecture/#distributor) validates incoming metric names and label names during ingestion. | ||||||
|
|
||||||
| When `name_validation_scheme` is set to `legacy`, Cortex applies the legacy validation behavior. When it is set to `utf8`, the Distributor accepts names that are valid under the UTF-8 scheme. | ||||||
|
|
||||||
| In practice, this means that enabling UTF-8 support can change whether samples or series are accepted during ingestion by the Distributor. With legacy validation, samples containing invalid metric or label names are dropped during validation. With UTF-8 validation enabled, names that are valid under the UTF-8 scheme can pass validation. Writers that send UTF-8 metric or label names require the Distributor to run with `utf8` validation enabled. | ||||||
|
|
||||||
| ## Recommended rollout strategy | ||||||
|
|
||||||
| 1. Enable `utf8` in a staging environment first. | ||||||
| 2. Test metric ingestion through the Distributor with representative UTF-8 metric names and label names. | ||||||
| 3. If your environment also relies on UTF-8 names in rule evaluation or alerting configuration, test the affected Ruler or Alertmanager workflows before production rollout. | ||||||
| 4. Roll out the selected validation scheme consistently wherever it is required. | ||||||
|
|
||||||
| ## Compatibility notes | ||||||
|
|
||||||
| - Use `legacy` if you need to preserve the previous validation behavior. | ||||||
| - Use `utf8` if you want Cortex to accept UTF-8 metric names and label names. | ||||||
| - Plan migration carefully when different systems in your environment assume different naming rules. | ||||||
| - Refer to the [configuration reference](https://cortexmetrics.io/docs/configuration/configuration-file/#supported-contents-and-default-values-of-the-config-file) for the current flag and supported values. | ||||||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's necessary to leave it in the changelog.