feat: rows-changed counter, flagged by codegen rather than counted by the opcode (#692) - #694
Open
dpsiderius wants to merge 1 commit into
Open
feat: rows-changed counter, flagged by codegen rather than counted by the opcode (#692)#694dpsiderius wants to merge 1 commit into
dpsiderius wants to merge 1 commit into
Conversation
… the opcode (#692) Nothing in `src/vdbe/` reported how many rows an `INSERT`/`UPDATE`/ `DELETE` changed. Spec 013 calls this the one item on its list a consumer cannot work around: `execute_transaction_step` returns rows and the autocommit flag, so a caller cannot tell an `UPDATE` that matched from one that did not, and that is the distinction every optimistic-concurrency scheme is built on. SQE swaps a table's metadata pointer with a conditional `UPDATE` and treats zero rows affected as a lost race; without the count that becomes SELECT-then-UPDATE, sound only while the consumer guarantees a single writer. The obvious implementation is wrong, and measurably so. Counting in the `Insert`/`Delete` handlers reports, per changed row: INSERT Insert -> 1 DELETE Delete -> 1 UPDATE, single-pass Delete + Insert -> 2 UPDATE, two-pass range-seek ephemeral Insert + Delete + Insert -> 3 The two-pass plan is #666/#675's range-seek path, which stashes matched rowids in an ephemeral b-tree using the same `Opcode::Insert`. So one `UPDATE` reports 2 or 3 depending on which plan the optimizer picked, and neither is 1. Index maintenance is the same shape: a write next to a row that is not a row change. So codegen marks the one mutation that counts, with `OPFLAG_NCHANGE` (`0x01`) on `P5` — stock SQLite's flag, same bit, same job. `P5` was unread by both opcodes, so nothing had to move, and no new opcode means the frozen-set ADRs (0015/0018/0020) stay closed. An `UPDATE` flags its `Insert` and not the paired `Delete`: one changed row, counted once. `StepOutcome::changes` is `Option<u64>`, and the two cases are not the same answer. `Some(0)` is "this was a DML statement and it changed nothing" — the lost race. `None` is "not that kind of statement", so a connection tracking `sqlite3_changes()` leaves its stored count alone after a `SELECT`. `Program::counts_changes()` is the discriminator and is deliberately *static*: an `UPDATE` whose `WHERE` matches nothing never executes its flagged `Insert` but must still report `Some(0)`. `execute_transaction_step` is now a wrapper over `execute_transaction_step_counted`, per ADR-0040's pattern — one loop, the old signature expressed in terms of the new one, so they cannot drift and its ten existing call sites are untouched. ADR-0042 records all of it, including why `u64` instead of `Option<u64>` would be a bug rather than a simplification. Both wrong designs are mutation-checked, not just argued: - counting unconditionally in the handlers fails `update_of_one_row_reports_one_under_both_plans`, `index_maintenance_does_not_count` and `conditional_update_reports_match`; - additionally flagging `UPDATE`'s `Delete` fails the same three, and fails the oracle diff with 4 against the oracle's 2. Verified: 1569 unit tests (1562 baseline + 7) and 381 corpus (380 + 1) pass, clippy/fmt/mod-files clean, assurance 86/86 and 276/276 with no dead links. `tests/corpus/changes_oracle_test.rs` diffs a thirteen-statement sequence against the pinned 3.53.4 oracle's own `changes()`, covering both `UPDATE` plans, a miss, a partial `DELETE` and a full one. Not included: `Connection::changes` and the cross-statement retention rule. A `Vm` lives for one statement so it cannot own that rule; it is spec 013/Req 1's surface and belongs to the facade ticket, where it is one line — store on `Some`, ignore `None`. One note for whoever merges second: `Execution` (#683) should grow a `changes()` the same way, which is a three-line addition once both are on `main`. The requirement IDs cited here live in #678, not yet on `main`. Refs: 013/Req-1, #692, #678, #683 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Nothing in
src/vdbe/reported how many rows anINSERT/UPDATE/DELETEchanged. Spec 013 calls this the one item on its list a consumer cannot
work around:
execute_transaction_stepreturns rows and the autocommitflag, so a caller cannot tell an
UPDATEthat matched from one that did not,and that distinction is what every optimistic-concurrency scheme is built on.
SQE swaps a table's metadata pointer with a conditional
UPDATEand treatszero rows affected as a lost race. Without the count that becomes
SELECT-then-
UPDATEinside a transaction — sound only while the consumerguarantees a single writer, and every consumer reinvents it.
The obvious implementation is wrong, and measurably so
Counting in the
Insert/Deletehandlers reports this, per changed row, onthe tree at 0.18.10:
INSERTInsertDELETEDeleteUPDATE, single-passDelete+Insert(update.rs:603,605)UPDATE, two-pass range-seekInsert(update.rs:276) +Delete+InsertThe two-pass plan is #666/#675's range-seek path, which stashes matched
rowids in an ephemeral b-tree using the same
Opcode::Insert. So the sameUPDATEreports 2 or 3 depending on which plan the optimizer picked, andneither is 1. Index maintenance is the same shape in reverse:
IdxInsert,IdxDeleteandAutoIndexInsertare writes adjacent to a row that are not arow change.
The opcode does not carry enough information to answer. Codegen does.
Design:
OPFLAG_NCHANGEon P5, as SQLite does itCodegen marks the one mutation that is the row change, with a
P5bit —stock SQLite's
OPFLAG_NCHANGE, same value (0x01), same job. Following itkeeps our opcode semantics aligned with the thing we are a replication of.
P5was unread by bothcursor::insertandcursor::delete, so the bitwas free on exactly the two opcodes that needed it. No new opcode, so the
frozen-set ADRs (0015/0018/0020) stay closed. An
UPDATEflags itsInsertand not the paired
Delete: one changed row, counted once.Some(0)andNoneare different answersStepOutcome::changesisOption<u64>:Some(0)— "this was anINSERT/UPDATE/DELETEand it changednothing." The lost race. This is the case the requirement exists to make
visible.
None— "not that kind of statement", so a connection trackingsqlite3_changes()leaves its stored count alone rather than zeroing it.Program::counts_changes()is the discriminator, and it is deliberatelystatic — it asks whether the program contains a flagged instruction,
not whether one executed. An
UPDATEwhoseWHEREmatches nothing neverruns its flagged
Insertbut must still reportSome(0).ADR-0042 records why collapsing this to a bare
u64would be a bug ratherthan a simplification: it merges "changed nothing" into "not a counting
statement", and then a
SELECTzeroes a count that should have survived it.No call-site churn
execute_transaction_stepbecomes a wrapper overexecute_transaction_step_counted, which returns the count. Same patternADR-0040 settled for streaming: one loop, the older signature expressed in
terms of the newer one, so they cannot drift. Its ten existing call sites
across
src/bin/, tests, benches and examples are untouched.Both wrong designs are mutation-checked, not just argued
update_of_one_row_reports_one_under_both_plans,index_maintenance_does_not_count,conditional_update_reports_matchUPDATE'sDeleteThe second one is the useful evidence that the oracle test is really talking
to the oracle rather than skipping green.
Test plan
tests/unit/vdbe_changes_test.rs— 7 tests. The load-bearing one isupdate_of_one_row_reports_one_under_both_plans, which builds bothplans (perf: UPDATE range-seek always pays for a two-pass deferred-rowid plan it usually doesn't need #675's rule: two-pass when
SETtouches the scanned index,single-pass when it does not) and asserts 1 from each
tests/corpus/changes_oracle_test.rs— a thirteen-statement sequencediffed against the pinned 3.53.4 oracle's own
changes(): bothUPDATEplans, a miss, a partialDELETE, a full one, and aDELETEon an already-empty tablecargo test --locked— 1569 passed / 0 failed (1562 baseline + 7)cargo test --locked --test corpus— 381 (380 + 1)make lint— both clippy passes, including the second one naming thetest = falsetargets where the new corpus test lives;cargo fmt --checkandmake check-mod-filescleanmake assurance— 86/86, 276/276, no dead links23bdc93from a clean detached worktree, not theworking tree
Notes for a reviewer
Connection::changesis still absent. This is the engine half. AVmlives for one statement, so it cannot own
sqlite3_changes()'scross-statement retention rule — that is the connection's, and belongs to
spec 013/Req 1's surface. What the facade has left to do is one line: store
on
Some, ignoreNone.SELECTis asserted viaProgram::counts_changes, not through theentry point.
compile_statementhandles write and DDL statements only; aSELECTreaches the engine by a different route (compile_select*+execute_with_db) that has no count to clobber in the first place. Thestatic discriminator is what a facade will consult, so it is what the test
pins.
P5onInsertis now meaningful where its doc comment previouslysaid conflict-resolution flags were "not modeled". A future
OR REPLACE/OR IGNOREimplementation must pick bits other than0x01.Execution(feat: streaming Execution primitive, with run() as its wrapper (#683) #691) should grow achanges()the same way — three linesonce both are on
main. Whichever merges second.013/Req-1exists only there.CHANGELOG.mdentry or version bump: folded separately per chore: fold #663 into 0.18.9, revert premature 0.18.10 bump #671.Spend: matched the
smallestimate. The design work was finding that thehandler-level count is plan-dependent, which cost one afternoon of reading
update.rsrather than any implementation effort.Refs: 013/Req-1, #678, #683
Closes #692
🤖 Generated with Claude Code