Add deployment-managed core migration primitives - #2437
Conversation
🦋 Changeset detectedLatest commit: ffba941 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
View logs |
docs | 96ba1b5 | Aug 12 2026, 02:04 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-cache | 0489cd4 | Aug 13 2026, 11:27 AM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
View logs |
emdash-demo-do | 96ba1b5 | Aug 12 2026, 02:04 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | 0489cd4 | Aug 13 2026, 11:27 AM |
Scope checkThis PR changes 3,358 lines across 51 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/auth-atproto
@emdash-cms/blocks
@emdash-cms/cloudflare
@emdash-cms/contentful-to-portable-text
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/plugin-cli
@emdash-cms/plugin-types
@emdash-cms/registry-client
@emdash-cms/registry-lexicons
@emdash-cms/registry-verification
@emdash-cms/sandbox-workerd
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-field-kit
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
There was a problem hiding this comment.
Pull request overview
Adds the core building blocks for deployment-managed core database migrations in packages/core, including a runtime policy (auto/check/manual), a validated secret-free migration manifest written at build/sync time, direct migration executors for SQLite/libSQL/Postgres, and public exports (identity + exact status) intended for deployment tooling.
Changes:
- Introduces runtime migration modes and enforces them during runtime initialization + middleware handling.
- Adds migration manifest primitives (identity, manifest schema/validation, manifest writer, config loader, integration metadata) and writes
.emdash/migrations.jsonduringastro build/astro sync. - Adds direct migration executors for SQLite/libSQL/Postgres plus a new exact migration status API (
knownApplied/pending/unknownApplied) with broad test coverage.
Reviewed changes
Copilot reviewed 50 out of 51 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/core/tsdown.config.ts | Adds new migration-related entrypoints to the build. |
| packages/core/tests/utils/test-db.ts | Exposes exact migration status helper for dialect tests. |
| packages/core/tests/unit/migrations/manifest.test.ts | Unit coverage for manifest build + validation (schema, secrets, entrypoints). |
| packages/core/tests/unit/migrations/manifest-writer.test.ts | Unit coverage for atomic manifest writes and failure cleanup. |
| packages/core/tests/unit/migrations/identity.test.ts | Unit coverage for identity fingerprinting and immutability. |
| packages/core/tests/unit/migrations/direct-executor.test.ts | Unit coverage for direct executor behavior, lifecycle, and failure modes. |
| packages/core/tests/unit/migrations/config-loader.test.ts | Unit coverage for config discovery/evaluation + metadata extraction + i18n normalization. |
| packages/core/tests/unit/i18n/normalize.test.ts | Unit coverage for Astro i18n normalization helper. |
| packages/core/tests/unit/db/migration-executors.test.ts | Unit coverage for sqlite/libsql/postgres migration executor modules. |
| packages/core/tests/unit/db/migration-adapters.test.ts | Unit coverage for adapter migration descriptors and secret separation. |
| packages/core/tests/unit/database/migration-policy.test.ts | Unit coverage for migration config normalization + runtime enforcement query counts. |
| packages/core/tests/unit/astro/setup-dev-bypass.test.ts | Ensures dev-bypass/setup routes don’t run migrations directly. |
| packages/core/tests/unit/astro/middleware-migration-policy.test.ts | Ensures middleware returns generic 503 behavior on pending migrations/manual mode. |
| packages/core/tests/unit/astro/integration/migration-policy.test.ts | Ensures integration rejects invalid migration policy modes. |
| packages/core/tests/unit/astro/integration/migration-manifest.test.ts | Ensures build/sync write secret-free manifests and metadata is attached correctly. |
| packages/core/tests/integration/runtime/create.test.ts | Ensures check mode doesn’t trigger migration-failure backoff. |
| packages/core/tests/integration/database/migrations.test.ts | Adds integration coverage for exact migration status behavior + query counts. |
| packages/core/tests/integration/database/dialect-compat.test.ts | Ensures exact migration status works across dialects. |
| packages/core/src/virtual-modules.d.ts | Extends virtual config typing to include migrations config. |
| packages/core/src/migrations/target.ts | Adds safe target fingerprinting + env var resolution helper. |
| packages/core/src/migrations/protocol.ts | Defines executor protocol types (request/target/report/factory). |
| packages/core/src/migrations/manifest.ts | Adds manifest v1 schema, secret-free validation, and serialization. |
| packages/core/src/migrations/manifest-writer.ts | Writes validated manifest atomically to .emdash/migrations.json. |
| packages/core/src/migrations/manifest-builder.ts | Builds a manifest from identity + i18n + adapter migration metadata. |
| packages/core/src/migrations/integration-metadata.ts | Attaches secret-free migration metadata to the Astro integration (symbol-keyed). |
| packages/core/src/migrations/index.ts | Exports migration identity, protocol, and direct executor APIs. |
| packages/core/src/migrations/identity.ts | Adds core migration identity + fingerprint generation. |
| packages/core/src/migrations/direct-executor.ts | Implements single-use direct executor (check/apply) with i18n scoping. |
| packages/core/src/migrations/config-loader.ts | Adds config-file discovery and safe evaluation fallback to build manifests from config. |
| packages/core/src/index.ts | Exports exact migration status + MIGRATION_NAMES/types from the public entry. |
| packages/core/src/i18n/normalize.ts | Adds normalization of Astro i18n input into EmDash’s I18nConfig. |
| packages/core/src/emdash-runtime.ts | Enforces runtime migration policy during DB init; avoids backoff for pending-migration errors. |
| packages/core/src/db/sqlite-migrations.ts | Adds deployment executor module for SQLite migrations. |
| packages/core/src/db/postgres-migrations.ts | Adds deployment executor module for Postgres migrations. |
| packages/core/src/db/libsql-migrations.ts | Adds deployment executor module for libSQL migrations. |
| packages/core/src/db/index.ts | Exposes exact migration status and MIGRATION_NAMES via db exports. |
| packages/core/src/db/adapters.ts | Adds adapter-provided migration metadata (secret-free manifestConfig). |
| packages/core/src/database/migrations/runner.ts | Adds MIGRATION_NAMES + exact migration status query helper. |
| packages/core/src/database/migrations/policy.ts | Adds migration config normalization + runtime enforcement for auto/check/manual. |
| packages/core/src/database/index.ts | Exposes exact migration status and MIGRATION_NAMES from database exports. |
| packages/core/src/astro/routes/api/setup/index.ts | Removes direct migration running from setup route. |
| packages/core/src/astro/routes/api/setup/dev-bypass.ts | Removes direct migration running from dev-bypass route (runtime now governs). |
| packages/core/src/astro/routes/api/auth/dev-bypass.ts | Removes direct migration running from auth dev-bypass route. |
| packages/core/src/astro/middleware.ts | Resolves effective migration mode and returns 503s for pending/manual-missing-schema cases. |
| packages/core/src/astro/integration/runtime.ts | Adds migrations config to EmDash integration config type. |
| packages/core/src/astro/integration/index.ts | Normalizes migration config; writes migration manifest during build/sync; attaches migration metadata. |
| packages/core/package.json | Exposes new public subpath exports for migration executors and ./migrations. |
| .changeset/runtime-migration-policy.md | Changeset for runtime migration policy modes. |
| .changeset/direct-migration-adapters.md | Changeset for deployment migration executors on built-in adapters. |
| .changeset/deployment-migration-primitives.md | Changeset for public migration identity + exact status APIs. |
| .changeset/deployment-migration-manifest.md | Changeset for manifest emission during build/sync. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![emdashbot[bot]](https://avatars.githubusercontent.com/in/3255304?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR introduces a well-scoped set of deployment-managed migration primitives: a runtime migration policy (auto/check/manual), build-time manifest generation and writing, identity/fingerprinting, and direct SQLite/libSQL/PostgreSQL executors. The approach fits EmDash’s architecture—writer runs at Astro build/sync time, executors are loaded from the same published subpaths used by the runtime, and the runtime enforces the policy without adding queries to the logged-out hot path beyond the existing migration fast-path count.
I read the new migration source, the adapter migration entrypoints, the integration wiring, the runtime policy integration, the Virtual-module type updates, the package exports, tsdown entries, and the test coverage. The code is generally clean, uses Kysely for SQL safety, keeps credentials out of the build artifact, and adds real behavioral tests for check/apply, secret rejection, target fingerprinting, config loading, and manual-mode middleware behavior.
One real logic bug stands out: the direct executor discards a successful migration report if closing the dialect fails. I also have one minor testing nit about a hard-coded SHA fingerprint literal.
github-actions
Bot
added
review/awaiting-author
Overlapping PRsThis PR modifies files that are also changed by other open PRs:
This may cause merge conflicts or duplicated work. A maintainer will coordinate. |
96ba1b5 to
4ccb0d3
Compare
github-actions
Bot
added
review/needs-rereview
There was a problem hiding this comment.
This PR adds a well-scoped set of deployment-managed migration primitives: a runtime migration policy (auto/check/manual), build-time manifest generation and writing, migration identity/fingerprinting, and direct SQLite/libSQL/PostgreSQL executors loaded from the same published subpaths the runtime uses. The approach fits EmDash’s architecture—writer runs at Astro build/sync time, the runtime enforces the policy without new logged-out hot-path queries beyond the existing migration fast-path count, and credentials are kept out of the build artifact. It links to the approved Discussion (#2418) and includes changesets, tests, and the necessary package/tsdown entries.
I read the new migration source, the adapter entrypoints, the integration and middleware wiring, the runtime policy integration, the Virtual-module type updates, the package exports, tsdown entries, and the test coverage. This is a static review; I did not run the test suite or build.
Prior-review status:
- ✅ Fixed: the direct executor no longer discards a successful migration report when closing the dialect fails; it now returns the report while logging the close error.
- 🔁 Still open: the hard-coded SHA-256 fingerprint literal in
identity.test.tsremains.
New finding: the middleware’s 503 response tells operators to run emdash migrate, but the EmDash CLI does not expose a migrate subcommand yet. That guidance is currently unfulfilled and will confuse operators when check or manual mode blocks traffic.
Verdict: comment — one real needs_fixing issue (the CLI message), one testing suggestion, and otherwise a clean, deliberate implementation.
github-actions
Bot
added
review/awaiting-author
github-actions
Bot
added
review/needs-rereview
There was a problem hiding this comment.
This PR lands the core primitives for deployment-managed migrations and addresses the concrete issues from the last review. The architecture is still the right fit for EmDash: build-time manifest generation keeps secrets out of the artifact, the runtime enforces auto/check/manual policy without adding new logged-out hot-path queries, and the direct SQLite/libSQL/PostgreSQL executors reuse the same published subpaths the runtime uses. It links to the approved Discussion (#2418), includes changesets, and wires the new subpaths through package.json and tsdown.config.ts.
I re-read the migration identity/manifest/executor code, the three adapter migration entrypoints, the integration wiring, the runtime policy integration, the middleware 503 path, and the new/updated tests.
Prior-review status:
- ✅ Fixed: the direct executor’s
finallynow logs a close failure and still returns a successful migration report. - ✅ Fixed: the middleware 503 message no longer references a non-existent
emdash migratesubcommand; it now tells operators to apply the deployment migration manifest. - 🔁 Still open: the hard-coded SHA-256 fingerprint literal in
identity.test.tsremains.
New observation: manual mode on a completely fresh database does not block traffic with the intended 503. Because EmDashRuntime.create swallows missing-table errors during its cold-start reads, runtime initialization succeeds; downstream DB queries (e.g. from getSiteSettings()) then fail with a raw 500. The middleware already has a branch for this case, but it is effectively unreachable for the fresh-DB scenario. This is a low-severity edge case because manual mode is opt-in, but it is worth an explicit decision or a targeted schema probe before traffic is served.
Headline: one carried-forward testing suggestion and one new operational edge case; otherwise the implementation is deliberate and clean.
There was a problem hiding this comment.
This is the right change for #2276/#2418: build-time migration manifests keep secrets out of the artifact, runtime auto/check/manual modes are wired through the existing middleware/runtime path without adding logged-out hot-path queries, and the direct SQLite/libSQL/PostgreSQL executors reuse the same published subpaths the runtime uses.
Re-reading the diff with prior review context in mind:
- Fixed since last review: the direct executor’s
finallycloses the database and still returns a successful report when close fails; the middleware 503 message now correctly tells operators to apply the deployment migration manifest; and manual mode on a fresh database now blocks traffic with a 503 becauseEmDashRuntime.createcaptures missing-table errors during cold-start reads and re-throws them. - Still open: the hard-coded SHA-256 fingerprint literal in
identity.test.tsremains. It is a config-pin test that restates the implementation rather than proving behavior.
I checked the new migration identity/manifest/executor code, the three adapter migration entrypoints, the integration manifest wiring, the runtime policy integration, the middleware 503 paths, and the new/updated tests. Conventions look good: SQL is parameterized/identifier-safe, changesets are present, no admin UI strings are added, and the first-request setup probe is limited to auto mode (the snapshots report no change).
github-actions
Bot
added
review/awaiting-author
ffba941 to
0489cd4
Compare
github-actions
Bot
added
review/needs-rereview
There was a problem hiding this comment.
This is the right change for #2276/#2418. The build-time migration manifest keeps runtime secrets out of the artifact, auto/check/manual runtime modes are wired into the existing middleware/runtime path without adding logged-out hot-path queries, and the direct SQLite/libSQL/PostgreSQL executors reuse the same published subpaths the runtime uses. The prior round’s fixes hold up: the direct executor still returns a successful report when close fails, the middleware 503 message correctly points operators to the deployment migration manifest, and manual mode on a fresh database now blocks traffic because EmDashRuntime.create captures missing-table errors during cold-start reads and re-throws them.
I re-read the migration identity/manifest/executor code, the three adapter migration entrypoints, the integration manifest wiring, the runtime policy integration, the middleware 503 paths, and the new/updated tests. Conventions look good: SQL is parameterized/identifier-safe, changesets are present and user-facing, no admin UI strings are added, and the setup probe is auto-only so the query-count snapshot stays unchanged.
One issue from the previous review is still present and should be fixed before merge:
- The fingerprint test hard-codes a SHA-256 literal. That is a config-pin test: it restates the implementation and cannot fail on any real regression other than a deliberate algorithm change. The preceding assertions already cover determinism and hex formatting, so the literal should be removed.
No new security, data-integrity, or logged-out query regressions were introduced.
| expect(first).toBe("ddb0be4cdff7aa2c58b5aee17e947043c7bbcacaa5043c8dbfe3db494ebdbf58"); | ||
| expect(second).toBe(first); | ||
| expect(first).toMatch(/^[0-9a-f]{64}$/); |
There was a problem hiding this comment.
[needs fixing] This assertion hard-codes the SHA-256 output for a single input. Per AGENTS.md, that is a config-pin test: it re-states the implementation and will only fail when someone intentionally changes the hashing method. The determinism and hex-format properties are already covered by expect(second).toBe(first) and the /^[0-9a-f]{64}$/ check, so the literal adds no behavioral signal.
| expect(first).toBe("ddb0be4cdff7aa2c58b5aee17e947043c7bbcacaa5043c8dbfe3db494ebdbf58"); | |
| expect(second).toBe(first); | |
| expect(first).toMatch(/^[0-9a-f]{64}$/); | |
| expect(second).toBe(first); | |
| expect(first).toMatch(/^[0-9a-f]{64}$/); |
github-actions
Bot
added
review/awaiting-author
What does this PR do?
Adds the core primitives for deployment-managed migrations: migration policies, manifests, configuration loading, direct SQLite/libSQL/PostgreSQL executors, runtime enforcement, and package exports. It also adds upgrade, policy, manifest, adapter, and executor coverage.
Part of #2276
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change)pnpm formathas been runmessages.pochanges except in translation PRs — a workflow extracts catalogs on merge tomain.Approved design discussion: #2418. Package typechecking, quick lint, and targeted formatting pass. The core suite passes 5,418 tests; its remaining failure is the pre-existing macOS
/varversus/private/varpath-normalization assertion.AI-generated code disclosure
Screenshots / test output
Non-visual change. Core package build and typecheck pass, query-count snapshots are unchanged, and the relevant migration tests pass.