From a3150958dee84285f147702b40e5fe3b13f2283c Mon Sep 17 00:00:00 2001 From: Peter Corke Date: Sun, 23 Aug 2026 08:56:18 +1000 Subject: [PATCH 1/4] fix(ci): keep JupyterLite wasm build on JSPI-safe cp312, not cibuildwheel's new cp313 default cibuildwheel's 3.4.1->4.1.1 bump silently moved the pyodide platform's default build target from cp312/Pyodide 0.27.7 to cp313/Pyodide 0.29.4, breaking docs-build once a real release exercised it. Adapting to the new default (and its PyPI-accepted pyemscripten_* tag) was tried and reverted same-day: Pyodide only gained cp313 support at 0.28.0a1, by which point WebAssembly JSPI was already on by default, and Safari has no JSPI support (Firefox only behind a flag) -- so cp312/0.27.x is the newest JSPI-safe line, matching MVTB's and bdsim's existing pins. Forces CIBW_BUILD: cp312-pyodide_wasm32 explicitly, restores the GitHub-Release-asset delivery mechanism (PyPI still rejects the old pyodide_* tag this produces), and fetches spatialgeometry's equivalent cp312 wheel from its own GitHub Release too, since RTB 1.4.0 stopped vendoring spatialgeometry as a pure-Python copy. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/ci.yml | 93 +++++++++++++++++++++++++---------- .github/workflows/release.yml | 23 +++++++-- pyproject.toml | 15 ++++++ 3 files changed, 101 insertions(+), 30 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ab83912a8..3bb6f5a97 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -188,34 +188,64 @@ jobs: # A plain `pip wheel`/`build` here would produce a native # macOS/Linux wheel with the compiled _fknm_c/_frne_c extensions -- # useless in Pyodide (WASM). release.yml already cross-compiles a - # genuine wasm32 wheel via cibuildwheel (CIBW_PLATFORM=pyodide) and - # attaches it to each GitHub Release (PyPI rejects the pyodide_* - # platform tag), so reuse that rather than rebuilding here. This - # means the "Try it Now" experience tracks the latest release, not - # bleeding-edge main -- deliberate, since a full Pyodide/Emscripten - # rebuild on every docs push would be slow. + # genuine wasm32 wheel via cibuildwheel (CIBW_PLATFORM=pyodide), + # forced to cp312 (see release.yml's build_pyodide job and + # pyproject.toml's [tool.cibuildwheel.pyodide] comment) and + # attaches it to each GitHub Release, since PyPI rejects this old, + # pre-standard `pyodide_*` platform tag. This means the "Try it + # Now" experience tracks the latest *release*, not bleeding-edge + # main -- deliberate, since a full Pyodide/Emscripten rebuild on + # every docs push would be slow. # - # STOPGAP (see https://github.com/petercorke/robotics-toolbox-python/issues/579): - # only fetch the cp312 wheel, matching - # jupyterlite-pyodide-kernel==0.6.1 below. A release can carry wasm - # wheels for more than one CPython version (cibuildwheel's pyodide - # platform builds one per supported interpreter), and + # Deliberately staying on cp312/the old tag rather than the newer + # PEP-783 `pyemscripten_*` tag PyPI now accepts: Pyodide only + # supports cp313+ from 0.28.0a1 onward, by which point WebAssembly + # JSPI ("stack switching") was already on by default (since + # 0.27.7) -- and Safari has no JSPI support, Firefox only behind a + # flag. cp312/0.27.x is the newest JSPI-safe line. See the + # toolbox-maintainer skill's JupyterLite version-pinning section + # before "modernizing" this away. + # + # The `--pattern '*cp312*pyodide*'` filter also guards against a + # real upstream bug (https://github.com/petercorke/robotics-toolbox-python/issues/579): # jupyterlite-pyodide-kernel's all.json generator keys entries by - # (name, version) and *overwrites* rather than accumulates, so - # fetching more than one wheel for the same package+version - # silently drops all but the alphabetically-last from the piplite - # index -- no build error, just a runtime "can't find a pure - # Python 3 wheel" in the browser. Confirmed still present in - # jupyterlite_pyodide_kernel 0.7.1 (latest as of 2026-07-21), not - # just 0.6.1 -- an upstream bug to route around, not a version to - # bump past. Intended to be removed once the pure-Python wasm - # wheel work lands and this whole fetch step goes away. + # (name, version) and *overwrites* rather than accumulates, so if a + # release ever carried wasm wheels for more than one CPython + # version, fetching more than one would silently drop all but the + # alphabetically-last from the piplite index -- no build error, + # just a runtime "can't find a pure Python 3 wheel" in the browser. + # release.yml's CIBW_BUILD restriction means only one wasm wheel + # (cp312) is ever built now, so this shouldn't trigger in practice, + # but the filter costs nothing to keep as a defence-in-depth guard. env: GH_TOKEN: ${{ github.token }} run: | mkdir -p docs/lite/pypi gh release download --repo "$GITHUB_REPOSITORY" --pattern '*cp312*pyodide*' --dir docs/lite/pypi --clobber + - name: Fetch spatialgeometry pyodide wheel for JupyterLite + # spatialgeometry is an unconditional runtime dependency of + # roboticstoolbox-python (since it stopped vendoring a pure-Python + # copy internally at 1.4.0), so `piplite.install` needs a wasm + # wheel for it too, staged in the same local index. Fetched from + # spatialgeometry's own GitHub Release (jhavl/spatialgeometry) -- + # it doesn't publish this to PyPI at all (tech debt raised + # upstream: jhavl/spatialgeometry#46, though note that issue's + # "publish via the new PEP-783 tag" suggestion has the same + # JSPI/Safari problem as RTB's own wheel above if it means moving + # off cp312). Matching RTB's own wheel above, this fetches the + # cp312 build specifically. Low-risk despite the cross-repo fetch: + # spatialgeometry's own pyproject.toml notes its CMake build skips + # the compiled nanobind extension entirely under Emscripten, so + # this wheel is genuinely pure Python -- no Pyodide ABI-version + # coupling to track here, unlike the JSPI/cp312 discipline the + # RTB wheel above needs. + env: + GH_TOKEN: ${{ github.token }} + run: | + gh release download --repo jhavl/spatialgeometry \ + --pattern '*cp312*pyodide*wasm32*' --dir docs/lite/pypi --clobber + - name: Sync notebooks into JupyterLite content # Copies *.ipynb plus any local asset directories (e.g. # docs/notebooks/figs/, referenced by relative path from markdown @@ -257,12 +287,23 @@ jobs: PY - name: Build JupyterLite site - # jupyterlite-pyodide-kernel is pinned deliberately: newer versions - # bundle a newer Pyodide runtime than the cp312 wasm wheel attached - # to releases targets, which manifests as a silent hang in the - # browser (no error, just a kernel that never responds) rather than - # a build-time failure. See toolbox-infrastructure.md for the full - # version-skew story before bumping this. + # jupyterlite-pyodide-kernel is pinned deliberately -- NOT just for + # wasm-ABI matching with the cp312 wheel fetched above, but because + # newer kernel releases bundle Pyodide >=0.27.7, which turned + # WebAssembly JSPI ("stack switching") on by default for + # run_until_complete(). Browsers without JSPI support (Safari has + # none as of 2026-08; Firefox only behind a flag) either crash the + # kernel outright or, with the enableRunUntilComplete: false + # workaround, turn run_until_complete into a fire-and-forget no-op -- + # letting a cell's package/import machinery run before an install has + # actually finished landing (confirmed live on bdsim: intermittent + # ModuleNotFoundError despite a successful-looking install cell). + # jupyterlite-pyodide-kernel==0.6.1 bundles Pyodide 0.27.6, genuinely + # predating this whole mechanism -- matches MVTB's and bdsim's + # existing pins. See the toolbox-maintainer skill's JupyterLite + # version-pinning section for the full story before bumping this. + # Once Safari (and Firefox out-of-flag) ship JSPI, revisit this pin + # across RTB/MVTB/bdsim together. run: | pip install jupyterlite-core "jupyterlite-pyodide-kernel==0.6.1" jupyter-server cd docs/lite && jupyter lite build --output-dir ../build/html/lite diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index fe2b4b696..3f761263e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -72,12 +72,22 @@ jobs: python3 --version pip3 --version pip3 show cibuildwheel 2>/dev/null || echo "cibuildwheel not installed locally (normal)" - echo "PYODIDE_VERSION=${{ vars.PYODIDE_VERSION || '0.27.5' }}" - name: Build Pyodide wheel + # Forced to cp312 deliberately -- Pyodide only ships JSPI-default-on + # (WebAssembly "stack switching") starting at 0.27.7, and Pyodide's + # cp313 support only starts at 0.28.0a1, i.e. *after* JSPI became the + # default. There is no JSPI-safe cp313 (or later) Pyodide build. + # Safari has no JSPI support; Firefox only behind a flag. Without + # this override, cibuildwheel 4.1.1's own default silently targets + # cp313 instead of cp312 -- exactly what broke this job once already + # (2026-08-23). See pyproject.toml's [tool.cibuildwheel.pyodide] + # comment and the toolbox-maintainer skill's JupyterLite + # version-pinning section for the full story before changing this. uses: pypa/cibuildwheel@v4.1.1 env: CIBW_PLATFORM: pyodide + CIBW_BUILD: cp312-pyodide_wasm32 - uses: actions/upload-artifact@v7 if: success() @@ -139,14 +149,19 @@ jobs: name: sdist path: dist - # Exclude Pyodide wheels — PyPI rejects the pyodide_* platform tag. - # They are uploaded as release assets in the next step instead. + # Exclude the Pyodide wheel -- PyPI rejects the old, pre-standard + # `pyodide_*` platform tag this cp312 build produces. Deliberately + # staying on this tag (not the newer PEP-783 `pyemscripten_*` one) to + # keep the wasm build JSPI-safe for Safari/Firefox -- see the + # `build_pyodide` job's comment and the toolbox-maintainer skill's + # JupyterLite version-pinning section. It's attached as a GitHub + # Release asset in the next job instead. - name: Remove Pyodide wheels from PyPI upload set run: rm -f dist/*pyodide*.whl - uses: pypa/gh-action-pypi-publish@release/v1 with: - skip_existing: true + skip-existing: true upload_pyodide_asset: name: Attach Pyodide wheel to GitHub release diff --git a/pyproject.toml b/pyproject.toml index a232825eb..1cb870b56 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -178,6 +178,21 @@ archs = ["AMD64"] # Pyodide (wasm32-emscripten) — used by JupyterLite # cibuildwheel manages the Emscripten SDK; scikit-build-core picks up # the toolchain file automatically from the PYODIDE environment. +# +# Pinned to cp312's default explicitly (matching release.yml's +# CIBW_BUILD: cp312-pyodide_wasm32 override) rather than left to +# cibuildwheel's own default, which is cp313/0.29.4 as of 4.1.1 -- that +# silently broke docs-build's JupyterLite step once already (2026-08-23). +# This is deliberately NOT the latest Pyodide cibuildwheel can target: +# Pyodide only ships cp313+ support from 0.28.0a1 onward, by which point +# JSPI ("stack switching") was already on by default (since 0.27.7) -- +# and Safari has no JSPI support, Firefox only behind a flag. cp312/0.27.x +# is the newest JSPI-safe line. jupyterlite-pyodide-kernel's pin in +# ci.yml must match (jupyterlite-pyodide-kernel==0.6.1, bundling Pyodide +# 0.27.6) -- see that file's "Fetch pyodide wheel for JupyterLite" step, +# and the toolbox-maintainer skill's JupyterLite version-pinning section +# for the full JSPI/Safari story before changing either pin. +pyodide-version = "0.27.7" [tool.pytest.ini_options] From 2243fc2990197baa5e60d53e49d8d32af8daa185 Mon Sep 17 00:00:00 2001 From: Peter Corke Date: Sun, 23 Aug 2026 09:04:48 +1000 Subject: [PATCH 2/4] fix(ci): add CIBW_ENABLE=pyodide-eol, required to build the cp312 pyodide target cibuildwheel 4.1.1 considers cp312/Pyodide 0.27.x "end of life" now that cp313/0.29.x is its new default, and refuses to build it at all without this explicit opt-in. Confirmed directly via a real workflow_dispatch run: without this, the job fails immediately with "Invalid build selector: 'cp312-pyodide_wasm32'. This selector matches a group that wasn't enabled." Co-Authored-By: Claude Sonnet 5 --- .github/workflows/release.yml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3f761263e..ee95922bb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -84,10 +84,18 @@ jobs: # (2026-08-23). See pyproject.toml's [tool.cibuildwheel.pyodide] # comment and the toolbox-maintainer skill's JupyterLite # version-pinning section for the full story before changing this. + # + # CIBW_ENABLE: pyodide-eol is required too -- cibuildwheel 4.1.1 + # considers cp312/0.27.x "end of life" now that cp313/0.29.x is its + # new default, and refuses to build it at all without this explicit + # opt-in (confirmed directly: omitting it fails outright with + # "Invalid build selector: 'cp312-pyodide_wasm32'. This selector + # matches a group that wasn't enabled"). uses: pypa/cibuildwheel@v4.1.1 env: CIBW_PLATFORM: pyodide CIBW_BUILD: cp312-pyodide_wasm32 + CIBW_ENABLE: pyodide-eol - uses: actions/upload-artifact@v7 if: success() From e428935b9fdb8c33647bbcfb1e011999a147eb3a Mon Sep 17 00:00:00 2001 From: Peter Corke Date: Sun, 23 Aug 2026 09:09:38 +1000 Subject: [PATCH 3/4] fix(ci): pin cibuildwheel 3.4.1 for the pyodide build job, not 4.1.1 The tag format pyodide-build emits (pyemscripten_* vs the old pyodide_*) turns out to be a property of the pyodide-build tool version bundled with cibuildwheel, not of which Pyodide version is targeted. Confirmed directly: cibuildwheel 4.1.1 forced onto cp312/Pyodide 0.27.7 still produces a pyemscripten_*-tagged wheel, which Pyodide 0.27.6's micropip (like every pre-PEP-783 release) hard-rejects as an unrecognized tag -- there is no pyodide-version override that fixes this on 4.1.1. cibuildwheel 3.4.1 predates the tag-format change and defaults to cp312/0.27.7 unprompted, producing the old pyodide_* tag that Pyodide 0.27.6 actually understands -- matches spatialgeometry's own pyodide build (same cibuildwheel version, same underlying reason). Drops the now-unnecessary CIBW_BUILD/CIBW_ENABLE overrides. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/release.yml | 40 +++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ee95922bb..9e3b26bea 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -74,28 +74,32 @@ jobs: pip3 show cibuildwheel 2>/dev/null || echo "cibuildwheel not installed locally (normal)" - name: Build Pyodide wheel - # Forced to cp312 deliberately -- Pyodide only ships JSPI-default-on - # (WebAssembly "stack switching") starting at 0.27.7, and Pyodide's - # cp313 support only starts at 0.28.0a1, i.e. *after* JSPI became the - # default. There is no JSPI-safe cp313 (or later) Pyodide build. - # Safari has no JSPI support; Firefox only behind a flag. Without - # this override, cibuildwheel 4.1.1's own default silently targets - # cp313 instead of cp312 -- exactly what broke this job once already - # (2026-08-23). See pyproject.toml's [tool.cibuildwheel.pyodide] + # Pinned to cibuildwheel 3.4.1 deliberately -- NOT the same version + # as the native wheel builds below (4.1.1) -- because the tag + # *format* pyodide-build emits turns out to be a property of the + # pyodide-build tool version, not of which Pyodide version it + # targets. Confirmed directly: forcing cibuildwheel 4.1.1 to build + # cp312/Pyodide 0.27.7 (via CIBW_BUILD + CIBW_ENABLE overrides) + # still produced a wheel tagged `pyemscripten_2024_0_wasm32` (the + # new PEP-783 tag), which Pyodide 0.27.6's micropip -- like every + # pre-PEP-783 Pyodide release -- hard-rejects with "Wheel was built + # with Emscripten vpyemscripten.2024.0 but Pyodide was built with + # Emscripten v3.1.58". There is no `pyodide-version` override that + # fixes this: cibuildwheel 4.1.1 cannot produce an old-tag wheel at + # all. cibuildwheel 3.4.1 predates the tag-format change and + # defaults to cp312/Pyodide 0.27.7 unprompted (matches + # spatialgeometry's own pyodide build, same version, same reason). + # + # This is why cp312/Pyodide 0.27.x matters at all: Pyodide only + # ships cp313+ support from 0.28.0a1 onward, by which point + # WebAssembly JSPI ("stack switching") was already on by default + # (since 0.27.7) -- and Safari has no JSPI support, Firefox only + # behind a flag. See pyproject.toml's [tool.cibuildwheel.pyodide] # comment and the toolbox-maintainer skill's JupyterLite # version-pinning section for the full story before changing this. - # - # CIBW_ENABLE: pyodide-eol is required too -- cibuildwheel 4.1.1 - # considers cp312/0.27.x "end of life" now that cp313/0.29.x is its - # new default, and refuses to build it at all without this explicit - # opt-in (confirmed directly: omitting it fails outright with - # "Invalid build selector: 'cp312-pyodide_wasm32'. This selector - # matches a group that wasn't enabled"). - uses: pypa/cibuildwheel@v4.1.1 + uses: pypa/cibuildwheel@v3.4.1 env: CIBW_PLATFORM: pyodide - CIBW_BUILD: cp312-pyodide_wasm32 - CIBW_ENABLE: pyodide-eol - uses: actions/upload-artifact@v7 if: success() From 394d258f9e65acebc94374eb1bf93fde980c2fd6 Mon Sep 17 00:00:00 2001 From: Peter Corke Date: Sun, 23 Aug 2026 09:13:09 +1000 Subject: [PATCH 4/4] fix(ci): drop the explicit pyodide-version override, let cibuildwheel 3.4.1 self-resolve Confirmed via a real build failure: cibuildwheel 3.4.1's bundled pyodide-build (0.33.0) rejects an explicit pyodide-version = "0.27.7" override as "not compatible", even though 0.27.7 is that same tool's own documented default for cp312. Matches spatialgeometry's own pyproject.toml, which never overrides this either. Co-Authored-By: Claude Sonnet 5 --- pyproject.toml | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 1cb870b56..905dd5ecc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -179,20 +179,27 @@ archs = ["AMD64"] # cibuildwheel manages the Emscripten SDK; scikit-build-core picks up # the toolchain file automatically from the PYODIDE environment. # -# Pinned to cp312's default explicitly (matching release.yml's -# CIBW_BUILD: cp312-pyodide_wasm32 override) rather than left to -# cibuildwheel's own default, which is cp313/0.29.4 as of 4.1.1 -- that -# silently broke docs-build's JupyterLite step once already (2026-08-23). -# This is deliberately NOT the latest Pyodide cibuildwheel can target: -# Pyodide only ships cp313+ support from 0.28.0a1 onward, by which point -# JSPI ("stack switching") was already on by default (since 0.27.7) -- -# and Safari has no JSPI support, Firefox only behind a flag. cp312/0.27.x -# is the newest JSPI-safe line. jupyterlite-pyodide-kernel's pin in -# ci.yml must match (jupyterlite-pyodide-kernel==0.6.1, bundling Pyodide -# 0.27.6) -- see that file's "Fetch pyodide wheel for JupyterLite" step, -# and the toolbox-maintainer skill's JupyterLite version-pinning section -# for the full JSPI/Safari story before changing either pin. -pyodide-version = "0.27.7" +# Deliberately NO explicit `pyodide-version` override here -- the +# `build_pyodide` job in release.yml is pinned to cibuildwheel 3.4.1 +# specifically (not the 4.1.1 used for native wheels) so it self-resolves +# its own default (cp312/Pyodide 0.27.7), matching spatialgeometry's own +# config exactly. An explicit `pyodide-version = "0.27.7"` override was +# tried and failed here: cibuildwheel 3.4.1's bundled pyodide-build +# (0.33.0) rejected it outright as "not compatible" even though it's +# that same tool's own documented default -- the override and the tool's +# internal default-resolution logic aren't equivalent, and only the +# latter actually works. Don't add one back without testing against a +# real build first. +# +# Why cp312/0.27.x specifically: Pyodide only ships cp313+ support from +# 0.28.0a1 onward, by which point WebAssembly JSPI ("stack switching") +# was already on by default (since 0.27.7) -- and Safari has no JSPI +# support, Firefox only behind a flag. cp312/0.27.x is the newest +# JSPI-safe line. jupyterlite-pyodide-kernel's pin in ci.yml must match +# (jupyterlite-pyodide-kernel==0.6.1, bundling Pyodide 0.27.6) -- see +# that file's "Fetch pyodide wheel for JupyterLite" step, and the +# toolbox-maintainer skill's JupyterLite version-pinning section for the +# full JSPI/Safari story before changing any of this. [tool.pytest.ini_options]