Skip to content

[P3][kernel] Explore an opt-in group-commit (fsync-batching) durability mode #62

Description

@cevheri

Summary

Roadmap item — the one remaining direction from the exploratory durability-hardening issue #9 (its other two directions, the versioned WAL header and the expanded DST fault profiles, shipped in v0.2.0 / PR #43). The kernel fsyncs on every commit: correct, simple, and the right DEFAULT — but per-commit fsync caps write throughput. The standard answer is group commit: several commits share one fsync, trading a bounded, explicitly documented durability window for throughput (cf. Postgres synchronous_commit, SQLite WAL).

Ground rules (carried from #9, still binding)

  • Opt-in only; the default stays fsync-per-commit. Nothing may quietly weaken the default durability contract in docs/RELIABILITY.md.
  • The mode's exact guarantee must be spelled out: what a crash can lose (commits acknowledged inside the open window) and when the window closes.
  • Must stay honest with the synchronous single-process model and interact correctly with the v0.2.0 failure latch (a failed batched fsync must latch exactly like a failed per-commit fsync, and the affected commits must be reported as NOT durable).

Open questions

  • API surface: an open() option (durability: "batched" + flush() fence), or an explicit db.batch(run) scope? The second keeps the window visible in the code that accepts it.
  • Crash semantics with the v1 record format: batching appends multiple records between fsyncs — recovery already tolerates a torn multi-record tail, but the DST needs the reordering fault profile (Explore kernel durability hardening (WAL record headers, fsync-batching mode, expanded DST fault profiles) #9 direction 3's deferred piece) to PROVE it: writes/fsyncs reaching durable storage out of order.
  • Whether acknowledged-but-unfsynced commits should be observable (a pending() count) so callers can build their own fences.

Deliverables

  1. A DESIGN.md addendum ratifying (or rejecting) the mode and its exact contract — documentation is as much the point as implementation (Explore kernel durability hardening (WAL record headers, fsync-batching mode, expanded DST fault profiles) #9's words).
  2. If ratified: implementation behind the opt-in, SimFS reordering profile, DST seeds proving the bounded-loss contract, RELIABILITY.md section for the mode.
  3. Changeset (minor) if shipped.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions