fix(subscriptions): Attribute subscription queries to a real organization id - #8400
fix(subscriptions): Attribute subscription queries to a real organization id#8400phacops wants to merge 6 commits into
Conversation
Subscription queries without an organization tenant id were backfilled with organization id 1 before query settings were built, so adding org 1 to max_groups_final_bypass_org_ids would have disabled the FINAL fallback for every unattributed subscription query rather than for that org alone. The placeholder is now applied during attribution only, after query settings are built, so it cannot drive per-organization query behavior.
Entities with an organization subscription processor persist the real organization id in subscription metadata, and RPC subscriptions already read it. Use it for SnQL subscription attribution too, so those queries carry their real organization instead of falling back to the placeholder.
Persist the organization for entities without an organization subscription processor, so their stored subscriptions stay attributable, and read it through a single helper instead of indexing metadata directly. Use 0 rather than 1 as the placeholder organization id, matching the querylog sentinel for an unknown organization so missing attribution is visible and can never collide with a real org.
tenant_ids values are typed str | int, so a bool organization id is not a case worth guarding against.
Sending a real organization for every subscription is handled in getsentry/sentry#122733, so the placeholder now only covers subscriptions stored before that change.
Name the field Sentry newly sends organization_id rather than organization, since it holds an id. The metrics entities' subscription processors stay configured with the legacy organization key, which every subscription stored so far carries and which AddColumnCondition raises on when missing. Give AddColumnCondition a fallback key so those entities accept either spelling, and read both when resolving the organization id.
| def get_organization_id(metadata: Mapping[str, Any]) -> int | None: | ||
| """Return the organization id from subscription metadata, if it has one.""" | ||
| organization_id = metadata.get( | ||
| ORGANIZATION_ID_METADATA_KEY, metadata.get(LEGACY_ORGANIZATION_METADATA_KEY) | ||
| ) | ||
| return int(organization_id) if organization_id is not None else None |
There was a problem hiding this comment.
Bug: The get_organization_id function directly casts the organization ID from metadata to an int without error handling, which can cause a ValueError if malformed legacy data is loaded from Redis.
Severity: MEDIUM
Suggested Fix
Wrap the int() conversion in a try-except ValueError block. In the except block, handle the case of a non-numeric ID, for example, by logging the error and returning None or raising a more specific exception to be handled upstream. This will prevent crashes from malformed data stored in Redis.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: snuba/subscriptions/data.py#L91-L96
Potential issue: The `get_organization_id` function retrieves an organization ID from
subscription metadata and directly casts it to an integer using `int()`. When
subscriptions are deserialized from Redis via `from_dict`, the metadata is not validated
for type correctness. If legacy data in Redis contains a non-numeric string for
`organization_id` or `organization`, the `int()` cast will raise a `ValueError`. This
would cause a runtime error during critical operations like `build_request()`,
`to_dict()`, or `filter_subscriptions()`, which rely on this function.
Did we get this right? 👍 / 👎 to inform future reviews.
|
I'm kind of wondering here whether the |
|
Hum, very good point, and then those PRs are not needed. |
Follow-up to #8399, which made
max_groups_final_bypass_org_idsskip the max-groups fallback toFINALfor listed organizations. That option matches onquery_settings.organization_id, and subscription queries did not reliably carry a real one, so this makes the organization behind a query trustworthy before the option gets turned on.The placeholder no longer reaches query settings. Subscription payloads do not always carry an organization tenant id, so
SnQLSubscriptionData.build_requestbackfilledorganization_id: 1intotenant_idsbefore query settings were built. Adding org 1 to the bypass list would then have disabledFINALfor every unattributed subscription query rather than for that one organization — and org 1 is plausibly a real org. The placeholder now lives inupdate_attribution_info, which runs after_get_settings_object, so allocation policies still see an organization tenant id whilequery_settings.organization_idstaysNoneunless a real one is known.The placeholder is now 0, matching the querylog sentinel for an unknown organization. It can never collide with a real organization id, so a nonzero count of
organization = 0in querylog measures how many subscriptions still lack attribution.Subscriptions use the organization they already store. The metrics entities persist it in subscription metadata via their
AddColumnConditionprocessor, andRPCSubscriptionDataalready read it back. SnQL subscriptions now do the same, and entities without such a processor persist it too, so their stored subscriptions stay attributable. Reads go through oneget_organization_idhelper instead of indexing metadata directly — which also fixes a latentKeyErrorin the sliced-storage scheduler path for subscriptions stored without the key.AddColumnConditionaccepts a fallback key. getsentry/sentry#122733 sends this field asorganization_id, while the metrics entities' processors are configured with the olderorganizationand raise when their configured key is missing. Without a fallback, metrics subscriptions created after that change would throwInvalidQueryExceptionon every scheduled execution. The configured key and the emittedorg_idcondition are unchanged; the processors simply also accept the new spelling.Existing subscriptions keep their stored payloads until they are recreated, so attribution fills in over time rather than at deploy. Errors and transactions subscriptions stay unattributed until the Sentry PR ships, which is why the placeholder remains a fallback rather than becoming an error.
Merge before getsentry/sentry#122733: this must be deployed before Sentry starts sending
organization_id, otherwise metrics subscriptions hit the missing-key path above.