fix(core): replace better-sqlite3 with node:sqlite in the sqlite adapter

[gruntlord5]

What does this PR do?

Replaces better-sqlite3 with Node's built-in node:sqlite driver in the sqlite() adapter and the CLI's createDatabase(). A small better-sqlite3-compatible wrapper (node-sqlite-compat.ts, ~70 lines) lets Kysely's built-in SqliteDialect run unchanged — statement .reader is derived from StatementSync.columns(), so INSERT ... RETURNING keeps working.

This removes the native compiled dependency that breaks deployments when the environment and binary disagree: NODE_MODULE_VERSION rebuild errors after Node upgrades (#562, #582) and the glibc hard-blocker on shared hosting (#1833). Discussion: #402.

Notes:

• better-sqlite3 moves to a devDependency (existing tests still use it directly).
• The NODE_MODULE_VERSION rebuild-hint helper is removed (no native binary to rebuild).
• emdash now declares engines: node >=22.15 (unflagged node:sqlite + StatementSync.columns()).

Type of change

• Bug fix
• Feature (requires maintainer-approved Discussion)
• Refactor (no behavior change)
• Translation
• Documentation
• Performance improvement
• Tests
• Chore (dependencies, CI, tooling)

Scope note: this fixes the native-binary deployment failures (#1833, #562, #582) by replacing the driver rather than patching it, so the diff is larger than a typical bug fix — happy to reclassify if you would rather treat it as a refactor.

Checklist

• I have read CONTRIBUTING.md
• pnpm typecheck passes (no errors in changed files; remaining errors identical on main)
• pnpm lint passes
• pnpm test passes (4751 passed; 1 pre-existing failure identical on clean main)
• pnpm format has been run
• I have added/updated tests for my changes (unit tests for the wrapper incl. RETURNING reader semantics; the existing 85 database tests now also run through it)
• User-visible strings in the admin UI are wrapped for translation (n/a — no admin UI changes)
• I have added a changeset
• Links to Discussion: Native node:sqlite database adapter #402

AI-generated code disclosure

• This PR includes AI-generated code — model/tool: Claude Fable 5 (Claude Code)

Screenshots / test output

Full core suite: 4751 passed / 1 failed — the failure is a pre-existing macOS temp-path test issue, verified identical on clean main. End-to-end validation on fixtures/perf-site (sqlite+node target): emdash init + emdash seed + astro build all succeed, and the built server serves seeded pages correctly.

[changeset-bot]

🦋 Changeset detected

Latest commit: d8f835b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
emdash	Minor
@emdash-cms/cloudflare	Minor
@emdash-cms/sandbox-workerd	Patch
@emdash-cms/fixture-perf-site	Patch
@emdash-cms/perf-demo-site	Patch
@emdash-cms/cache-demo-site	Patch
@emdash-cms/do-demo-site	Patch
@emdash-cms/do-solo-demo-site	Patch
@emdash-cms/admin	Minor
@emdash-cms/auth	Minor
@emdash-cms/blocks	Minor
@emdash-cms/gutenberg-to-portable-text	Minor
@emdash-cms/x402	Minor
create-emdash	Minor
@emdash-cms/auth-atproto	Patch
@emdash-cms/plugin-embeds	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

Scope check

This PR changes 686 lines across 13 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

[emdashbot]

This is the right change for the problem described in #402. Moving the SQLite path from better-sqlite3 to Node's built-in node:sqlite removes a native compiled dependency that causes real deployment pain (NODE_MODULE_VERSION mismatches, glibc issues on shared hosting), and the small better-sqlite3-compat wrapper is a pragmatic way to keep Kysely's SqliteDialect unchanged.

I checked the wrapper, the CLI/runtime adapter wiring, the package changes, the Vite/tsdown externalization, and the tests. The approach is sound and the package-level changes (engines: node>=22.15, moving better-sqlite3 to devDependencies, updating NODE_NATIVE_EXTERNALS, the changeset) are all correct.

However, there are two issues that should be addressed before merge:

1. The test harness still runs on better-sqlite3. packages/core/tests/utils/test-db.ts creates its SQLite test DB with import Database from "better-sqlite3", so the existing database suite is not exercising the new node:sqlite/wrapper path. This contradicts the PR description's claim that "the existing 85 database tests now also run through it" and leaves the production driver under-tested. The harness should switch to openNodeSqliteDatabase(":memory:").

2. The parameter wrapper is too permissive for booleans. toBindings casts Kysely's values with as SQLInputValue[], but the SQLInputValue type excludes boolean. better-sqlite3 accepts JS booleans (stored as 0/1); node:sqlite rejects them at bind time, so any query with a boolean value will throw at runtime. The wrapper should map booleans explicitly.

A few other cleanup notes (non-blocking):

• pnpm-lock.yaml contains a lot of unrelated churn (removed top-level overrides/patchedDependencies, many hasBin/libc snapshot changes). If these are from an unintentional pnpm install regeneration, revert them to keep the change focused; if they're intentional, call them out in the PR.
• The runtime adapter in packages/core/src/db/sqlite.ts still doesn't enable WAL/foreign keys, while the CLI path in connection.ts does. This isn't a regression from the PR, but it's worth centralizing now that DB creation lives in one place.
• Several templates/demos/e2e fixtures still list better-sqlite3 as a dependency. Since they no longer need it at runtime, consider removing it so new sites don't pull the native binary.

[emdashbot]

[needs fixing] The new unit tests cover the wrapper in isolation, but the shared test harness in packages/core/tests/utils/test-db.ts still uses import Database from "better-sqlite3" for its SQLite databases. That means the existing database test suite is not actually running through the node:sqlite driver, which contradicts the PR description and leaves the production code path under-tested.

Switch the harness to create SQLite test databases through the wrapper so the full suite exercises the driver that ships with the package:

Suggested change

	describe("openNodeSqliteDatabase", () => {
	// In packages/core/tests/utils/test-db.ts
	import { openNodeSqliteDatabase } from "../../src/db/node-sqlite-compat.js";
	export function createTestDatabase(): Kysely<DatabaseSchema> {
	resetSchemaCachesForTests();
	const sqlite = openNodeSqliteDatabase(":memory:");
	return new Kysely<DatabaseSchema>({
	dialect: new SqliteDialect({
	database: sqlite,
	}),
	});
	}

[emdashbot]

[needs fixing] toBindings casts Kysely's parameters to SQLInputValue[] without any value mapping, but SQLInputValue excludes boolean. better-sqlite3 accepts JavaScript booleans and stores them as 0/1; node:sqlite rejects booleans at bind time, so any query that inserts, updates, or selects on a boolean value will throw a runtime TypeError. This is a real compatibility gap for any code that relied on the old driver's coercion.

Map booleans to integers before spreading them into the statement:

Suggested change

	function toBindings(parameters: ReadonlyArray<unknown>): SQLInputValue[] {
	return parameters.map((value) => {
	if (typeof value === "boolean") return value ? 1 : 0;
	return value as SQLInputValue;
	});
	}

[emdashbot]

The approach is right: replacing better-sqlite3 with Node's built-in node:sqlite directly addresses the deployment failures in #402, and the small compat wrapper is a pragmatic way to keep Kysely's SqliteDialect unchanged.

The author has resolved the two blockers from my prior review:

• packages/core/tests/utils/test-db.ts now creates databases through openNodeSqliteDatabase.
• node-sqlite-compat.ts now maps booleans to 0/1 and rejects Date parameters with an actionable error.
• Connection pragmas (WAL, busy_timeout, foreign_keys) are centralized in openNodeSqliteDatabase, so the runtime adapter and CLI share the same settings.
• better-sqlite3 was removed from all the templates/demos/e2e fixtures I pointed out, and the lockfile churn is now focused on the dependency move.

However, this re-review found three remaining gaps that should be addressed before merge:

1. Most tests still bypass the new driver. The shared harness is fixed, but I found ~30+ test files that still import Database from "better-sqlite3" and construct their own in-memory DBs. This contradicts the PR's claim that "the existing 85 database tests now also run through it" and leaves the production node:sqlite/wrapper path under-tested.
2. skills/wordpress-theme-to-emdash/scaffold/package.json still depends on better-sqlite3 at runtime. Sites generated from this skill will continue to pull the native binary, which defeats the PR's purpose for that entry path.
3. scripts/sync-templates-repo.mjs still emits better-sqlite3 in the standalone template allowBuilds config. Now that templates have removed the dependency, this stale allow-list will fail strictDepBuilds in the standalone templates repo.

I also flagged two small cleanup suggestions: unused better-sqlite3 devDependencies in packages/marketplace and a stale better-sqlite3 entry in knip.json for demos.

Static review only — I did not run the test suite, linter, or build.

---

Findings

• [needs fixing] packages/core/tests/utils/test-db.ts:47

  The shared harness now correctly uses openNodeSqliteDatabase, but this is the only DB creation path I found that was migrated. A search of the tree shows many test files still import better-sqlite3 directly — for example packages/core/tests/unit/plugins/state.test.ts:8, packages/core/tests/integration/loader-taxonomy-pivot-plan.test.ts:16, and packages/workerd/test/bridge-handler.test.ts:12. In each case they instantiate their own new Database(":memory:") rather than using this utility.

  That contradicts the PR checklist claim that "the existing 85 database tests now also run through it" and leaves the production node:sqlite/node-sqlite-compat.ts path under-tested. Any test that isn't explicitly simulating better-sqlite3/D1 quirks should switch to createTestDatabase() or openNodeSqliteDatabase(":memory:").

• [needs fixing] skills/wordpress-theme-to-emdash/scaffold/package.json:16

  better-sqlite3 is still listed as a runtime dependency in this scaffold template, so sites generated from the WordPress skill will continue to install the native binary that the PR is meant to remove. Since EmDash now uses node:sqlite, this dependency should be dropped from the scaffold as well.

  		"@astrojs/react": "^5.0.0-beta.1",
  		"astro": "^6.0.0-beta.0",
  		"emdash": "workspace:*",
  

• [needs fixing] scripts/sync-templates-repo.mjs:160-161

  pnpmWorkspaceYaml still emits better-sqlite3 entries for both Cloudflare and non-Cloudflare templates. The templates themselves no longer depend on better-sqlite3, so these allowBuild entries will fail strictDepBuilds for the standalone templates repo (or simply be irrelevant for Cloudflare). Remove them.

  	const allowBuilds = template.endsWith("-cloudflare")
  		? "  esbuild: true\\n  workerd: true\\n  sharp: false"
  		: "  esbuild: true\\n  sharp: true\\n  workerd: false";
  

• [suggestion] packages/marketplace/package.json:21-23

  @types/better-sqlite3 and better-sqlite3 are listed as devDependencies, but neither the source nor the tests import better-sqlite3. They appear to be unused leftovers from an earlier iteration. If they're not needed, remove them to keep the package lean. The same likely applies to packages/workerd once its test files are migrated off better-sqlite3.

• [suggestion] knip.json:28

  better-sqlite3 is still in demos/* ignoreDependencies, but all demos in the diff have dropped the dependency. This ignore is now stale and should be removed.

[github-actions]

This PR has been inactive for 14 days. It will be closed automatically in 7 days if there is no further activity.

If you're still working on this, please push an update or leave a comment.

[github-actions]

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

• Add deployment-managed core migration primitives #2437 (5 shared files)

This may cause merge conflicts or duplicated work. A maintainer will coordinate.