feat(pivot): warehouse-computed row totals for pivot tables#23780
Merged
Conversation
🧪 Test Selection✅ Tests that will run
⏭️ Tests skipped (no relevant file changes detected)
|
|
Docs PR opened: lightdash/mintlify-docs#740 Clarified that pivot table row totals, like column totals, are computed by the warehouse for accurate non-additive metrics. |
|
Your preview environment pr-23780 has been deployed. Preview environment endpoints are available at: |
Preview Environment🌐 URL: https://lightdash-preview-pr-23780.lightdash.okteto.dev 📋 Logs: View in GCP Console 🔧 SSH: |
ZeRego
force-pushed
the
pivot-table-row-totals
branch
from
d7e0c68 to
a884001
Compare
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
Backend: extends POST /api/v2/projects/{projectUuid}/query/{queryUuid}/calculate-total
with kind: 'rowTotal'. The source MetricQuery + PivotConfiguration are re-run
with groupByColumns cleared and only the index dims kept, so the warehouse
produces one correct total per index-value row (collapsed across all pivoted
columns) for every metric type. Row totals require a pivoted source; non-pivoted
sources are rejected.
Frontend: useAsyncCalculateRowTotal streams the flat totals query and keys each
row by its index-dim values (buildPivotRowTotalKey). useTableConfig feeds the
resulting map into the SQL-pivot worker.
Row totals are now EXCLUSIVELY warehouse-computed — there is no client-side
summation. The worker allocates a row-total slot for every visible base metric
(both metrics-as-columns and metrics-as-rows layouts) and fills it from the
warehouse map, leaving the cell blank (null) when no value is available
(loading, errored, or a source query that can't be totalled). This makes row
totals correct for count_distinct/avg/ratio metrics instead of summing
pre-aggregated cells, and stops showing wrong sums for non-additive metrics.
Key matching handles the wire shape the totals query returns: metric columns
carry the `<metric>_any` aggregation suffix, and date index values serialize
with milliseconds (`...00.000Z`) unlike the pivot query (`...00Z`), so the
lookup normalizes ISO datetimes and tries the suffixed column name.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ZeRego
force-pushed
the
pivot-table-row-totals
branch
from
a884001 to
697685b
Compare
lightdash-bot
pushed a commit
that referenced
this pull request
Jun 2, 2026
# [0.3080.0](0.3079.0...0.3080.0) (2026-06-02) ### Bug Fixes * **pivot:** alias multi-join CTE columns so ClickHouse resolves them downstream (PROD-8026) ([#23711](#23711)) ([#23790](#23790)) ([bda5e29](bda5e29)) ### Features * **ai:** searchSemanticLayer tool for project-wide metric inventory (PROD-8072) ([#23787](#23787)) ([7a4ae18](7a4ae18)) * **pivot:** warehouse-computed row totals for pivot tables ([#23780](#23780)) ([e3a3d10](e3a3d10))
Collaborator
|
🎉 This PR is included in version 0.3080.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: PROD-7974
Description:
Adds
kind: 'rowTotal'toPOST /api/v2/projects/{projectUuid}/query/{queryUuid}/calculate-total, which re-runs the source pivot query withgroupByColumnscleared and only the index dims kept, so the warehouse returns one correct total per index-value row for every metric type. The frontend (useAsyncCalculateRowTotal) streams those totals into the pivot worker, which now computes pivot-table row totals exclusively from the warehouse — no client-side cell summation — for both the metrics-as-columns and metrics-as-rows layouts. This fixes row totals for non-additive metrics (count distinct, average, ratios) that were previously wrong or hidden, leaving a cell blank when no warehouse value is available rather than showing an incorrect sum. This is the row-totals sibling of the merged column-totals PR #23590.Demos