Skip to content

fix(subscriptions): Attribute subscription queries to a real organization id - #8400

Closed
phacops wants to merge 6 commits into
masterfrom
fix/max-groups-final-bypass-subscriptions
Closed

fix(subscriptions): Attribute subscription queries to a real organization id#8400
phacops wants to merge 6 commits into
masterfrom
fix/max-groups-final-bypass-subscriptions

Conversation

@phacops

@phacops phacops commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #8399, which made max_groups_final_bypass_org_ids skip the max-groups fallback to FINAL for listed organizations. That option matches on query_settings.organization_id, and subscription queries did not reliably carry a real one, so this makes the organization behind a query trustworthy before the option gets turned on.

The placeholder no longer reaches query settings. Subscription payloads do not always carry an organization tenant id, so SnQLSubscriptionData.build_request backfilled organization_id: 1 into tenant_ids before query settings were built. Adding org 1 to the bypass list would then have disabled FINAL for every unattributed subscription query rather than for that one organization — and org 1 is plausibly a real org. The placeholder now lives in update_attribution_info, which runs after _get_settings_object, so allocation policies still see an organization tenant id while query_settings.organization_id stays None unless a real one is known.

The placeholder is now 0, matching the querylog sentinel for an unknown organization. It can never collide with a real organization id, so a nonzero count of organization = 0 in querylog measures how many subscriptions still lack attribution.

Subscriptions use the organization they already store. The metrics entities persist it in subscription metadata via their AddColumnCondition processor, and RPCSubscriptionData already read it back. SnQL subscriptions now do the same, and entities without such a processor persist it too, so their stored subscriptions stay attributable. Reads go through one get_organization_id helper instead of indexing metadata directly — which also fixes a latent KeyError in the sliced-storage scheduler path for subscriptions stored without the key.

AddColumnCondition accepts a fallback key. getsentry/sentry#122733 sends this field as organization_id, while the metrics entities' processors are configured with the older organization and raise when their configured key is missing. Without a fallback, metrics subscriptions created after that change would throw InvalidQueryException on every scheduled execution. The configured key and the emitted org_id condition are unchanged; the processors simply also accept the new spelling.

Existing subscriptions keep their stored payloads until they are recreated, so attribution fills in over time rather than at deploy. Errors and transactions subscriptions stay unattributed until the Sentry PR ships, which is why the placeholder remains a fallback rather than becoming an error.

Merge before getsentry/sentry#122733: this must be deployed before Sentry starts sending organization_id, otherwise metrics subscriptions hit the missing-key path above.

Subscription queries without an organization tenant id were backfilled with organization id 1 before query settings were built, so adding org 1 to max_groups_final_bypass_org_ids would have disabled the FINAL fallback for every unattributed subscription query rather than for that org alone.

The placeholder is now applied during attribution only, after query settings are built, so it cannot drive per-organization query behavior.
Entities with an organization subscription processor persist the real organization id in subscription metadata, and RPC subscriptions already read it. Use it for SnQL subscription attribution too, so those queries carry their real organization instead of falling back to the placeholder.
@phacops phacops changed the title fix(replacer): Keep placeholder org out of max-groups FINAL bypass fix(replacer): Attribute subscription queries to a real organization id Aug 26, 2026
Persist the organization for entities without an organization subscription processor, so their stored subscriptions stay attributable, and read it through a single helper instead of indexing metadata directly.

Use 0 rather than 1 as the placeholder organization id, matching the querylog sentinel for an unknown organization so missing attribution is visible and can never collide with a real org.
tenant_ids values are typed str | int, so a bool organization id is not a case worth guarding against.
Sending a real organization for every subscription is handled in getsentry/sentry#122733, so the placeholder now only covers subscriptions stored before that change.
Name the field Sentry newly sends organization_id rather than organization, since it holds an id. The metrics entities' subscription processors stay configured with the legacy organization key, which every subscription stored so far carries and which AddColumnCondition raises on when missing.

Give AddColumnCondition a fallback key so those entities accept either spelling, and read both when resolving the organization id.
@phacops phacops changed the title fix(replacer): Attribute subscription queries to a real organization id fix(subscriptions): Attribute subscription queries to a real organization id Aug 26, 2026
@phacops
phacops marked this pull request as ready for review August 26, 2026 19:11
@phacops
phacops requested review from a team as code owners August 26, 2026 19:12
Comment on lines +91 to +96
def get_organization_id(metadata: Mapping[str, Any]) -> int | None:
"""Return the organization id from subscription metadata, if it has one."""
organization_id = metadata.get(
ORGANIZATION_ID_METADATA_KEY, metadata.get(LEGACY_ORGANIZATION_METADATA_KEY)
)
return int(organization_id) if organization_id is not None else None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The get_organization_id function directly casts the organization ID from metadata to an int without error handling, which can cause a ValueError if malformed legacy data is loaded from Redis.
Severity: MEDIUM

Suggested Fix

Wrap the int() conversion in a try-except ValueError block. In the except block, handle the case of a non-numeric ID, for example, by logging the error and returning None or raising a more specific exception to be handled upstream. This will prevent crashes from malformed data stored in Redis.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: snuba/subscriptions/data.py#L91-L96

Potential issue: The `get_organization_id` function retrieves an organization ID from
subscription metadata and directly casts it to an integer using `int()`. When
subscriptions are deserialized from Redis via `from_dict`, the metadata is not validated
for type correctness. If legacy data in Redis contains a non-numeric string for
`organization_id` or `organization`, the `int()` cast will raise a `ValueError`. This
would cause a runtime error during critical operations like `build_request()`,
`to_dict()`, or `filter_subscriptions()`, which rely on this function.

Did we get this right? 👍 / 👎 to inform future reviews.

@MeredithAnya

Copy link
Copy Markdown
Member

I'm kind of wondering here whether the max_groups_final_bypass_org_ids should have been project ids instead? Replacements are specific to errors and the errors table doesn't have the org id, so the subscription queries aren't going to have the org id anyway. Delete requests are going to be scoped to a single project as well.

phacops commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Hum, very good point, and then those PRs are not needed.

@phacops phacops closed this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants