fix(customize): skip rows with a missing raw record instead of ending the scan - #9027
Open
vbhanuchander-lang wants to merge 1 commit into
Open
Conversation
… the scan The extractor cursor LEFT JOINs the raw table, so a domain row whose _raw_data_id points at a raw record that has since been deleted comes back with a NULL data column. That value fell through to the default branch of the type switch, which returned nil and ended the whole scan. Nothing surfaced: the subtask reported success, and every row ordered after the first orphan silently kept whatever custom field values it already had. The reporter saw 1,382 orphaned rows out of 30,287 leave roughly 28,000 rows unpopulated. Skip the row and carry on. extractCustomizedFields now returns how many rows it skipped for that reason, and the caller logs a warning naming the table and the count, so the condition is visible instead of silent. Also return rows.Err() at the end of the scan. An error raised while iterating the cursor was previously discarded, which is the same class of silent partial processing. Signed-off-by: Bhanu Chander Vallabaneni <v.bhanuchander@gmail.com>
vbhanuchander-lang
force-pushed
the
fix-customize-orphaned-raw-data
branch
from
August 1, 2026 19:23
3ecb2b7 to
9b0e585
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The customize extractor cursor LEFT JOINs the raw table, so a domain row whose
_raw_data_idpoints at a raw record that has since been deleted comes back with a NULLdatacolumn. That value fell through to the default branch of the type switch:Nothing surfaced. The subtask reported success, and every row ordered after the first orphan silently kept whatever custom field values it already had — so a partially-populated table looks identical to a correctly-populated one. In the reported case 1,382 orphaned rows out of 30,287 left roughly 28,000 rows unpopulated.
This skips the row and carries on.
Change
defaultnow increments a counter andcontinues rather than returning.extractCustomizedFieldsreturns how many rows it skipped for that reason, andExtractCustomizedFieldslogs a warning naming the table, the raw table and the count. A summary rather than per-row logging, since the count can run into the thousands.rows.Err(). An error raised while iterating the cursor was previously discarded, which is the same class of silent partial processing and would be invisible in exactly the same way.Skipping is the right behaviour rather than failing: an orphaned
_raw_data_idmeans the raw record was cleaned up, which is a normal consequence of raw-data retention, not a corrupt state. Failing the subtask would make routine cleanup break every later sync.Does this close any open issues?
Closes #8945
Screenshots
N/A — backend behaviour, no UI surface.
Other Information
Tests. The plugin had no unit tests, so this adds four, using the generated
dalmocks to drive a cursor: orphans interleaved with valid rows (the reported shape), all rows orphaned, no orphans, and a cancelled context.I checked the main test actually catches the bug rather than just passing — temporarily restoring the old
return nilmakes it fail with"[]" should have 2 item(s), but has 0, i.e. zero rows updated because the scan aborted at the first orphan. With the fix, both valid rows are updated and the two orphans are counted.go test ./plugins/customize/tasks/passes,gofmtandgo vetclean for the changed files. Theplugins/customize/e2epackage fails locally for want ofE2E_DB_URL, which is unrelated and excluded fromscripts/unit-test-go.sh.One thing I noticed but did not change. The
[]byteandstringbranches are not symmetric: thestringbranch has theissues+IsArray()handling that writesIssueCustomArrayFieldrows, and the[]bytebranch does not. Which branch a row takes depends on what the driver returns for the rawdatacolumn, so that feature may behave differently across databases. That felt like a separate question from this fix, and changing it could alter behaviour for existing users, so I left it alone — happy to open a follow-up issue if you would like it looked at.