test(core): pin that append-only reads do not apply deletes - #656
Closed
linliu-code wants to merge 1 commit into
Closed
test(core): pin that append-only reads do not apply deletes#656linliu-code wants to merge 1 commit into
linliu-code wants to merge 1 commit into
Conversation
The append-only merge strategy concatenates the data batches and returns, so the delete batches are never consulted — a deleted record is still present in the result. That is defensible for a table that is genuinely append-only, but the strategy is not opt-in: the table config derives it whenever meta fields are disabled or no ordering field is set. A merge-on-read table meeting either condition has log files that can carry delete blocks, and those deletes are dropped. No fixture in the repo reaches this — every merge-on-read fixture sets an ordering field and leaves meta fields on — which is why it had gone unnoticed. The test pins the behavior so a change to it is deliberate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 3, 2026
Contributor
Author
|
Closing: already contained in the surviving line.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Independent of the merge-on-read reader port — this is about the existing read path.
What I found
RecordMerger's append-only arm is:concat_data_batchesconcatenatesdata_batchesand returns.delete_batchesis never read. A deleted record is still in the result.Why that is reachable
Append-only is defensible for a table that genuinely only appends. But it is not opt-in —
config/table.rsderives it:hoodie.populate.meta.fields = false→append_only, orhoodie.table.ordering.fields/precombine.field→append_onlyA merge-on-read table meeting either condition has log files that can carry delete blocks. Those deletes are dropped, and updates are returned as duplicates rather than merged (the existing
test_merge_records_append_onlyalready documents the duplication — 4 rows out of 2+2 with a repeated key).Why it went unnoticed
I checked every merge-on-read fixture in the repo: all 14 set an ordering field and leave meta fields on, so none derives append-only. The path has no fixture coverage at all.
What this PR does
Adds a test pinning the current behavior, so a change to it is deliberate rather than accidental. It does not change behavior — I did not want to alter existing read results in a test-only PR, and the right fix is a design question:
Worth a maintainer's call. This came up because the ported merge-on-read reader has no append-only equivalent — it always merges by key — so the same table would return different rows under it.
🤖 Generated with Claude Code