Skip to content

ref(attrvalues): support array-of-string attributes and typed value_data - #8390

Open
MeredithAnya wants to merge 7 commits into
masterfrom
meredith/EAP-698
Open

ref(attrvalues): support array-of-string attributes and typed value_data#8390
MeredithAnya wants to merge 7 commits into
masterfrom
meredith/EAP-698

Conversation

@MeredithAnya

@MeredithAnya MeredithAnya commented Aug 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Adds array-of-string support to TraceItemAttributeValues, and returns values as
typed AttributeValues with per-value metadata instead of plain strings.

Depends on getsentry/sentry-protos#412 (released in
sentry-protos 0.64.1, which this PR pins).

What changed

  • TYPE_ARRAY_STRING keys are now enumerable. attributes_array_string is
    added to the type → column map, so requesting an array-of-string key returns
    its distinct array values instead of BadSnubaRPCRequestException.
  • New value_data field. Every value is returned as a
    ValueData { value: AttributeValue, count, last_seen }. Because the value is a
    typed AttributeValue, arrays come back whole (val_array) and booleans come
    back as real val_bool — neither of which the string-only response could
    express. Conversion reuses get_converter_for_type from the table resolver
    (renamed from _get_converter_for_type) so typing matches TraceItemTable.
  • New last_seen per value. The outer query selects
    max(timestamp) AS last_seen, with timestamp projected through the inner
    query. Like count(), this is an aggregate over the 10k-row sample the inner
    query takes, not over every matching item — noted in the docstring.
  • datetime coercion helper moved to common.py as as_datetime, shared
    with endpoint_trace_item_attribute_names.py (native driver returns a
    datetime, HTTP returns an ISO string).

Backwards compatibility

The deprecated values/counts fields are unchanged for string and boolean
keys — booleans keep the lowercase "true"/"false" form that callers feed
back in as filter inputs. Array keys populate only value_data: the string
fields cannot represent an array, and this endpoint rejected array keys before
value_data existed, so no caller can be relying on them. The page token is now
derived from value_data so array responses paginate correctly.

@linear-code

linear-code Bot commented Aug 25, 2026

Copy link
Copy Markdown

EAP-698

@MeredithAnya MeredithAnya mentioned this pull request Aug 25, 2026
Merged
@MeredithAnya MeredithAnya changed the title ref(attrvalues): enable attribute_array_string column ref(attrvalues): support array-of-string attributes and typed value_data Aug 26, 2026
@MeredithAnya
MeredithAnya marked this pull request as ready for review August 26, 2026 22:41
@MeredithAnya
MeredithAnya requested review from a team as code owners August 26, 2026 22:41
MeredithAnya
MeredithAnya commented Aug 26, 2026
View reviewed changes
Comment thread snuba/web/rpc/v1/trace_item_attribute_values.py Outdated
@MeredithAnya MeredithAnya mentioned this pull request Aug 27, 2026
Merged
MeredithAnya added a commit that referenced this pull request Aug 27, 2026
@MeredithAnya
ref(attribute-values): return the most recent items (#8409)
83140ff 
Related to #8390

We are going to be adding a `last_seen` to the
`TraceItemAttributeValuesResponse` (among other things) but right now
the inner query has no order by.

We want to add to order by timestamp so that the 10000 we grab are the
most recent, so that when we get the `last_seen` aka `max(timestamp)`,
this actually represents the most recent data.

However in order to get the benefit of `optimize_read_in_order` we need
the ORDER BY be a prefix of the primary key. Since we use `project_ids
IN ()` syntax, if we just have one project, order by timestamp doesn't
optimize the query. So you'd need the full prefix `(organization_id,
project_id, item_type, timestamp)` in the ORDER BY, and with multiple
projects that might not make sense since you'd be getting data from one
project

### examples 

**project_id = 1**

```EXPLAIN PIPELINE
SELECT
    attributes_string_38['sentry.description'],
    timestamp
FROM eap_items_1_local
WHERE (project_id = 1) AND (organization_id = 1) AND (item_type = 1)
ORDER BY timestamp DESC
LIMIT 10
SETTINGS optimize_read_in_order = 1

Query id: ec4ecaf9-9d49-46dd-aea7-6d566cc2dc62

    ┌─explain───────────────────────────────────────────────────────────────────────────────────┐
 1. │ (Expression)                                                                              │
 2. │ ExpressionTransform                                                                       │
 3. │   (Limit)                                                                                 │
 4. │   Limit                                                                                   │
 5. │     (Sorting)                                                                             │
 6. │     MergingSortedTransform 2 → 1                                                          │
 7. │       BufferChunks × 2                                                                    │
 8. │         (Expression)                                                                      │
 9. │         ExpressionTransform × 2                                                           │
10. │           (Expression)                                                                    │
11. │           ExpressionTransform × 2                                                         │
12. │             (ReadFromMergeTree)                                                           │
13. │             ReverseTransform                                                              │
14. │               MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InReverseOrder) 0 → 1     │
15. │                 ReverseTransform                                                          │
16. │                   MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InReverseOrder) 0 → 1 │
    └───────────────────────────────────────────────────────────────────────────────────────────┘

16 rows in set. Elapsed: 0.015 sec.
````

**project_id IN [1]**

```
EXPLAIN PIPELINE
SELECT
    attributes_string_38['sentry.description'],
    timestamp
FROM eap_items_1_local
WHERE (project_id IN [1]) AND (organization_id = 1) AND (item_type = 1)
ORDER BY timestamp DESC
LIMIT 10
SETTINGS optimize_read_in_order = 1

Query id: 89de6449-8eee-4381-b8a2-50dfe3a98628

    ┌─explain──────────────────────────────────────────────────────────────────────┐
 1. │ (Expression)                                                                 │
 2. │ ExpressionTransform                                                          │
 3. │   (Limit)                                                                    │
 4. │   Limit                                                                      │
 5. │     (Sorting)                                                                │
 6. │     MergingSortedTransform 2 → 1                                             │
 7. │       MergeSortingTransform × 2                                              │
 8. │         LimitsCheckingTransform × 2                                          │
 9. │           PartialSortingTransform × 2                                        │
10. │             (Expression)                                                     │
11. │             ExpressionTransform × 2                                          │
12. │               (Expression)                                                   │
  13. │               ExpressionTransform × 2                                        │
14. │                 (ReadFromMergeTree)                                          │
15. │                 MergeTreeSelect(pool: ReadPool, algorithm: Thread) × 2 0 → 1 │
    └──────────────────────────────────────────────────────────────────────────────┘

15 rows in set. Elapsed: 0.022 sec.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pbhandari pbhandari pbhandari approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MeredithAnya @pbhandari