Skip to content

feat: Switch to semble-grammars - #238

Merged
Pringled merged 8 commits into
mainfrom
grammars
Aug 3, 2026
Merged

feat: Switch to semble-grammars#238
Pringled merged 8 commits into
mainfrom
grammars

Conversation

@Pringled

@Pringled Pringled commented Aug 1, 2026

Copy link
Copy Markdown
Member

This PR swaps out tree-sitter-language-pack for https://github.com/MinishLab/semble-grammars. This means the binaries become ~8mb bigger, but we get:

  • No runtime downloads since all grammars are bundled, so there's no network calls after installing (except the one time model download)
  • Compiled language packs for all platforms (including Intel macs and Arm Windows builds which we didn't have before)
  • Control over what is and isn't packaged
  • Bit easier error handling
  • One less dependency

Pringled added 5 commits July 17, 2026 11:56
semble-grammars 0.1.1 is now on PyPI, so drop the local editable-path
override and pin a real version instead. uv.lock still resolves against
the local path for now; regenerating it needs semble-grammars to clear
the 1-week exclude-newer window (it was only just published).
…ad-free

The tri-state manual cache existed to memoize a fallible, network-based
download call. semble_grammars.get_parser bundles all grammars at install
time, so a plain functools.cache suffices, matching the pattern already
used in chunking/core.py's _cached_get_parser.
0.1.2 adds the embeddedtemplate alias, batch/heex grammars, and zsh alias
found while benchmarking this branch against tree-sitter-language-pack.
Adds an exclude-newer-package override for semble-grammars so uv.lock
can resolve it immediately instead of waiting out the general 1-week
recency guard, since we control its release cadence.
@codecov

codecov Bot commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/semble/chunking/chunking.py 100.00% <100.00%> (ø)
src/semble/chunking/core.py 100.00% <100.00%> (ø)
src/semble/installer/config.py 100.00% <100.00%> (ø)
src/semble/version.py 100.00% <100.00%> (ø)

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@greptile-apps

greptile-apps Bot commented Aug 1, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge with no concrete changed-code failure identified.

The new parser APIs preserve the existing fallback behavior, the installer handles unavailable JSON5 parsing, and the dependency replacement introduces no changed vulnerable dependency path.

Reviews (1): Last reviewed commit: "Regenerate uv.lock against real semble-g..." | Re-trigger Greptile

ALL_LANGUAGES is just the extension-to-language mapping table, so every
language value reaching chunk_source already satisfied this check by
construction. The real support check already happens one layer down:
chunk() returns None when _cached_get_parser() fails for any reason,
and chunk_source already falls back to line chunking in that case.

@stephantul stephantul left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIIIICE 🚀

Comment thread pyproject.toml Outdated
Per review: a fixed-date override needs manual bumping every time we
release. semble-grammars is ours, so the recency guard's supply-chain
rationale doesn't apply to it anyway.
@Pringled
Pringled merged commit 8eb1450 into main Aug 3, 2026
16 checks passed
@Pringled
Pringled deleted the grammars branch August 3, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants