ref(attribute-values): return the most recent items

[MeredithAnya]

Related to #8390

We are going to be adding a last_seen to the TraceItemAttributeValuesResponse (among other things) but right now the inner query has no order by.

We want to add to order by timestamp so that the 10000 we grab are the most recent, so that when we get the last_seen aka max(timestamp), this actually represents the most recent data.

However in order to get the benefit of optimize_read_in_order we need the ORDER BY be a prefix of the primary key. Since we use project_ids IN () syntax, if we just have one project, order by timestamp doesn't optimize the query. So you'd need the full prefix (organization_id, project_id, item_type, timestamp) in the ORDER BY, and with multiple projects that might not make sense since you'd be getting data from one project

examples

project_id = 1

SELECT
    attributes_string_38['sentry.description'],
    timestamp
FROM eap_items_1_local
WHERE (project_id = 1) AND (organization_id = 1) AND (item_type = 1)
ORDER BY timestamp DESC
LIMIT 10
SETTINGS optimize_read_in_order = 1

Query id: ec4ecaf9-9d49-46dd-aea7-6d566cc2dc62

    ┌─explain───────────────────────────────────────────────────────────────────────────────────┐
 1. │ (Expression)                                                                              │
 2. │ ExpressionTransform                                                                       │
 3. │   (Limit)                                                                                 │
 4. │   Limit                                                                                   │
 5. │     (Sorting)                                                                             │
 6. │     MergingSortedTransform 2 → 1                                                          │
 7. │       BufferChunks × 2                                                                    │
 8. │         (Expression)                                                                      │
 9. │         ExpressionTransform × 2                                                           │
10. │           (Expression)                                                                    │
11. │           ExpressionTransform × 2                                                         │
12. │             (ReadFromMergeTree)                                                           │
13. │             ReverseTransform                                                              │
14. │               MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InReverseOrder) 0 → 1     │
15. │                 ReverseTransform                                                          │
16. │                   MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InReverseOrder) 0 → 1 │
    └───────────────────────────────────────────────────────────────────────────────────────────┘

16 rows in set. Elapsed: 0.015 sec.

project_id IN [1]

EXPLAIN PIPELINE
SELECT
    attributes_string_38['sentry.description'],
    timestamp
FROM eap_items_1_local
WHERE (project_id IN [1]) AND (organization_id = 1) AND (item_type = 1)
ORDER BY timestamp DESC
LIMIT 10
SETTINGS optimize_read_in_order = 1

Query id: 89de6449-8eee-4381-b8a2-50dfe3a98628

    ┌─explain──────────────────────────────────────────────────────────────────────┐
 1. │ (Expression)                                                                 │
 2. │ ExpressionTransform                                                          │
 3. │   (Limit)                                                                    │
 4. │   Limit                                                                      │
 5. │     (Sorting)                                                                │
 6. │     MergingSortedTransform 2 → 1                                             │
 7. │       MergeSortingTransform × 2                                              │
 8. │         LimitsCheckingTransform × 2                                          │
 9. │           PartialSortingTransform × 2                                        │
10. │             (Expression)                                                     │
11. │             ExpressionTransform × 2                                          │
12. │               (Expression)                                                   │
  13. │               ExpressionTransform × 2                                        │
14. │                 (ReadFromMergeTree)                                          │
15. │                 MergeTreeSelect(pool: ReadPool, algorithm: Thread) × 2 0 → 1 │
    └──────────────────────────────────────────────────────────────────────────────┘

15 rows in set. Elapsed: 0.022 sec.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 542c4b5. Configure here.}