Skip to content

fix(content): search searchable custom fields - #2191

Open
logelog wants to merge 1 commit into
emdash-cms:mainfrom
logelog:feat/content-list-searchable-fields
Open

fix(content): search searchable custom fields#2191
logelog wants to merge 1 commit into
emdash-cms:mainfrom
logelog:feat/content-list-searchable-fields

Conversation

@logelog

@logelog logelog commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Content-list search now includes every schema field marked searchable, in addition to the collection display fields and slug.

The FTS path already indexed searchable custom fields, but the fallback path only matched slug, title, and name. That made custom metadata search backend-dependent and prevented values such as internal ticket numbers from being found when FTS could not serve the query.

This change keeps the existing FTS path, resolves searchable fields from the collection schema, and casts fallback values to text so scalar custom fields can be matched consistently across supported SQL dialects.

Addresses the search portion of #2179.

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). No user-visible strings were added.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: N/A (bug fix)

AI-generated code disclosure

  • This PR includes AI generated code: Claude Opus 5, GPT-5.6 (via private dev orchestra)

Screenshots / test output

  • pnpm --filter emdash test --run tests/integration/content/content-list-search.test.ts tests/integration/content/content-list-search-fts.test.ts (17 tests passed)
  • pnpm typecheck
  • pnpm lint
  • pnpm format:check

@changeset-bot

changeset-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1cf9d73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Jul 22, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@2191

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@2191

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@2191

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@2191

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@2191

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@2191

emdash

npm i https://pkg.pr.new/emdash@2191

create-emdash

npm i https://pkg.pr.new/create-emdash@2191

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@2191

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@2191

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@2191

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@2191

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@2191

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@2191

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@2191

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@2191

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@2191

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@2191

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@2191

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@2191

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@2191

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@2191

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@2191

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@2191

commit: 1cf9d73

@logelog
logelog force-pushed the feat/content-list-searchable-fields branch from 97a4de1 to be1120f Compare July 22, 2026 09:20
github-actions Bot pushed a commit that referenced this pull request Jul 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

This may cause merge conflicts or duplicated work. A maintainer will coordinate.

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for the described problem. The bug — that content-list search only matched slug/title/name in the non-FTS path while the FTS index already covered all searchable fields — is real, and the change resolves it by reading searchable flags from _emdash_fields and including those columns in both the FTS-coverage decision and the LIKE fallback. CASTing column values to TEXT makes the fallback work consistently across SQLite/Postgres for scalar custom fields (including the integer field tested), and validateIdentifier keeps the dynamic identifiers safe.

What I checked:

  • Diff and changed files: resolveSearchColumns now pulls searchable flags; applySearchFilter casts to TEXT for the LIKE fallback; docstrings and tests are updated.
  • AGENTS.md conventions: SQL identifiers are validated via validateIdentifier; no user-facing strings are added; a proper changeset is present; no new logged-out queries are introduced; comments are descriptive, not reviewer-facing or issue-referencing.
  • FTS consistency: canUseFtsForListFilter checks that every non-slug search column is present in the FTS index's searchable-field set, so the fallback still kicks in when a display field like title isn't searchable. The schema registry already rebuilds the FTS index when a field's searchable flag changes, so the index stays in sync with the new column set.
  • Tests: Both the FTS path and the LIKE fallback path now have regressions for searching a custom field, and the integer-field case is covered in the dialect-agnostic test. Tests are behavioral, not tautological.

Headline conclusion: clean, well-scoped bug fix. No blocking issues found.

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 27, 2026
@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/needs-review No maintainer or bot review yet labels Jul 27, 2026
@logelog
logelog force-pushed the feat/content-list-searchable-fields branch from 4d8e02c to 8990c44 Compare August 3, 2026 00:21
@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/approved Approved; no new commits since labels Aug 3, 2026
@logelog
logelog force-pushed the feat/content-list-searchable-fields branch from 8990c44 to 1cf9d73 Compare August 3, 2026 12:31
@MA2153 MA2153 added the bot:review Trigger an emdashbot code review on this PR label Aug 3, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for the reported gap. The content-list search previously matched only slug/title/name in the non-FTS path, while the FTS index already honored the searchable flag. The PR resolves the full set of searchable columns from _emdash_fields, includes them in both the FTS coverage decision and the LIKE fallback, and CASTs values to TEXT so scalar custom fields (including integers) are matched consistently across SQLite and Postgres.

What I checked:

  • Approach and correctness: resolveSearchColumns now adds every field with searchable === 1; canUseFtsForListFilter correctly falls back to LIKE when any non-slug display column is not covered by the FTS index. The CAST(... AS TEXT) change in applySearchFilter is standard SQL and works for both integer and text columns.
  • SQL safety: Dynamic identifiers are passed through validateIdentifier before sql.ref(), so user-defined field slugs cannot be injected.
  • Locale filtering: Content-table queries still apply locale before the search filter, and FTS rows are scoped to the same table.
  • Index sync: The schema registry rebuilds the FTS index whenever a field's searchable flag changes (createField, updateField, deleteField), so the index stays consistent with the new column set.
  • Logged-out query counts: handleContentList is gated on content:read; no anonymous visitor can trigger the new _emdash_fields query.
  • Tests: The new regressions cover both the FTS path and the LIKE fallback, including an integer field in the dialect-agnostic test. They are behavioral, not tautological.
  • Changeset: Concise, present-tense, and describes the user-visible effect.

I found one non-blocking stale docstring in FindManyOptions; the headline conclusion is still clean and ready to merge.


Findings

  • [suggestion] packages/core/src/database/repositories/types.ts:157

    The searchColumns docstring still says the columns come from the collection's "display fields", but after this change resolveSearchColumns also adds slug and every field marked searchable. The stale description can mislead callers about what the q filter will actually match.

    		 * Resolved by the handler from the collection's display fields, slug,
    		 * and any field marked searchable. Each name is validated as a SQL identifier.
    

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Aug 3, 2026
@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/needs-rereview Author pushed changes since the last review labels Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants