diff --git a/AGENTS.md b/AGENTS.md index 455e8de7..8f35c9e2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,7 @@ - **`Figure`** (`anyplotlib/figure/_figure.py`) — the only `anywidget.AnyWidget` subclass. Owns all traitlets and is the Python↔JS bridge. - **Plot objects** (`plot1d/`, `plot2d/`, `plot3d/`) — `Plot1D`, `PlotBar`, `Plot2D`, `PlotMesh`, `Plot3D` are **plain Python classes**, not widgets. They hold state in `_state` dicts and push to the Figure. Shared behaviour lives in `_base_plot.py` (`_BasePlot`, `_PanelMixin`, `_MarkerMixin`). - **`Axes`** (`axes/_axes.py`) — grid-cell container; factory methods (`imshow`, `plot`, `bar`, `pcolormesh`, `plot_surface`, …) create plot objects and attach them. -- **`figure_esm.js`** — pure-JS canvas renderer (~9,300 lines); all rendering logic lives here. **Read `anyplotlib/FIGURE_ESM.md` first** — it is the section map. +- **`figure_esm.js`** — pure-JS canvas renderer (~12,210 lines); all rendering logic lives here. **Read `anyplotlib/FIGURE_ESM.md` first** — it is the section map. - **`markers.py`** — static visual overlays (circles, arrows, lines, etc.) with a two-level dict registry: `plot.markers[type][name]`. - **`widgets/`** — interactive draggable overlays (`RectangleWidget`, `CrosshairWidget`, etc.) that receive JS position updates. - **`callbacks.py`** — event system: `Event` dataclass, `CallbackRegistry` (priority ordering, wildcard, pause/hold), `_EventMixin` (`add_event_handler`). @@ -115,7 +115,7 @@ grep -nE '^\s*(function|const|let) [A-Za-z_]' anyplotlib/figure_esm.js ``` and reconcile against the two numbered tables (the section map near the top and -the 2-D function table). Both were last verified at 10,119 lines. +the 2-D function table). Both were last verified at 12,211 lines. Changelog entries: add a fragment file to `upcoming_changes/` (e.g. `123.new_feature.rst`) — towncrier assembles `CHANGELOG.rst` at release time. @@ -129,7 +129,7 @@ Use `api_change` when existing behaviour changes, even if the change is a fix. | `anyplotlib/figure/_gridspec.py` | `GridSpec`, `SubplotSpec` | | `anyplotlib/figure/_subplots.py` | `subplots()` factory | | `anyplotlib/axes/_axes.py` | `Axes` — plot factory methods | -| `anyplotlib/figure_esm.js` | All JS canvas rendering (~9,300 lines) | +| `anyplotlib/figure_esm.js` | All JS canvas rendering (~12,210 lines) | | `anyplotlib/FIGURE_ESM.md` | Section map for `figure_esm.js` — read this before editing the JS | | `anyplotlib/markers.py` | Static marker collections; `to_wire()` translation | | `anyplotlib/widgets/` | Interactive overlay widgets | diff --git a/anyplotlib/FIGURE_ESM.md b/anyplotlib/FIGURE_ESM.md index 9f3f9023..da85c815 100644 --- a/anyplotlib/FIGURE_ESM.md +++ b/anyplotlib/FIGURE_ESM.md @@ -1,6 +1,6 @@ # FIGURE_ESM.md — Navigator for `figure_esm.js` -`figure_esm.js` is **~9,470 lines** and one big closure. Everything lives inside +`figure_esm.js` is **~12,210 lines** and one big closure. Everything lives inside `function render({ model, el })` so that all helpers share the same scope (`theme`, `PAD_*`, `panels` Map, etc.). This document is a section map so you can jump straight to the relevant code without reading the whole file. @@ -53,28 +53,28 @@ Rule 5 – Text never clips. Optional gutters earn real layout space: | b64 array decode helpers | 109 | | **Rich-text (mini-TeX) engine**: `_texRuns` / `_texLayout` / `_drawTex` | 161 / 228 / 250 | | **2D gutter geometry**: `_cbWidth` / `_cbGap` / `_padT` / `_titlePx` | 301 / 313 / 323 / 333 | -| **Layout engine** `applyLayout` | 778 | -| `_buildCanvasStack` | 861 | -| `_createPanelDOM` | 1003 | -| `_createInsetDOM` / `_applyAllInsetStates` | 1144 / 1538 | -| `_resizePanelDOM` | 2251 | -| **2D drawing**: `_imgFitRect` | 2415 | -| `draw2d` | 2744 | -| `drawScaleBar2d` / `drawColorbar2d` | 2939 / 3219 | -| **Floating keys**: `_keyEnsure` / `_keyRect` / `drawKeys` | 3038 / 3061 / 3074 | -| `_drawAxes2d` (ticks, labels, title) | 3273 | -| `drawOverlay2d` / `drawMarkers2d` | 3426 / 3590 | -| **Image layers**: `_layerBytes` / `_layerBitmap` / `_drawLayers2d` | 2564 / 2588 / 2649 | -| Binary-bytes splice: `_spliceBinaryBytes` / `_registerBinaryPixelListeners` | 734 / 765 | -| **Hover readout**: `_pixelValue2d` / `_readoutInfo2d` | 4434 / 4516 | -| `_notifyReadout` / `_updateStatus2d` / `_armValueProbe` | 4556 / 4571 / 4592 | -| **3D drawing**: `draw3d` | 5568 | -| Event emission `_emitEvent` | 6405 | -| 3D event handlers `_attachEvents3d` | 6462 | -| **1D drawing**: `draw1d` | 6686 | -| `_drawLine` (1D series + markers) | 6839 | -| `drawOverlay1d` / `drawMarkers1d` | 7132 / 7216 | -| Marker hit-test `_markerHitTest2d` | 7484 | +| **Layout engine** `applyLayout` | 824 | +| `_buildCanvasStack` | 907 | +| `_createPanelDOM` | 1049 | +| `_createInsetDOM` / `_applyAllInsetStates` | 1190 / 1584 | +| `_resizePanelDOM` | 2297 | +| **2D drawing**: `_imgFitRect` | 2461 | +| `draw2d` | 2790 | +| `drawScaleBar2d` / `drawColorbar2d` | 2985 / 3265 | +| **Floating keys**: `_keyEnsure` / `_keyRect` / `drawKeys` | 3084 / 3107 / 3120 | +| `_drawAxes2d` (ticks, labels, title) | 3319 | +| `drawOverlay2d` / `drawMarkers2d` | 3472 / 3636 | +| **Image layers**: `_layerBytes` / `_layerBitmap` / `_drawLayers2d` | 2610 / 2634 / 2695 | +| Binary-bytes splice: `_spliceBinaryBytes` / `_registerBinaryPixelListeners` | 780 / 811 | +| **Hover readout**: `_pixelValue2d` / `_readoutInfo2d` | 4480 / 4562 | +| `_notifyReadout` / `_updateStatus2d` / `_armValueProbe` | 4602 / 4617 / 4638 | +| **3D drawing**: `draw3d` | 5614 | +| Event emission `_emitEvent` | 6451 | +| 3D event handlers `_attachEvents3d` | 6508 | +| **1D drawing**: `draw1d` | 6732 | +| `_drawLine` (1D series + markers) | 6885 | +| `drawOverlay1d` / `drawMarkers1d` | 7178 / 7262 | +| Marker hit-test `_markerHitTest2d` | 7530 | > **`raster` marker (1D/PlotXY)** — `drawMarkers1d` has a `type==='raster'` > branch that blits a single RGBA image across data-coord `extent` (the fast @@ -83,20 +83,22 @@ Rule 5 – Text never clips. Optional gutters earn real layout space: > redraws never re-transmit them; the decoded `OffscreenCanvas` is cached on > the marker set (`ms._rasterBmp`/`_rasterKey`). The shared `clip_path` block > clips it to a curved sector. -| Panel event dispatch `_attachPanelEvents` | 7741 | -| 2D events `_attachEvents2d` | 7783 | -| 1D events `_attachEvents1d` | 8176 | -| 2D widget drag `_ovHitTest2d` / `_doDrag2d` | 8451 / 8730 | -| **Brush strokes**: `_brushLiveBegin` / `_brushCommit` / `_brushErase` / `_brushPaintAt` | 8643 / 8657 / 8686 / 8721 | -| 1D widget drag `_canvasXToFrac1d` … / snapping `_snapVal` | 8855 / 8928 | -| Shared-axis propagation `_getShareGroups` | 8999 | -| Figure resize `_applyFigResizeDOM` | 9063 | -| **Bar chart**: `_barGeom` / `drawBar` / `_attachEventsBar` | 9254 / 9317 / 9693 | -| Generic redraw `_redrawPanel` | 9883 | -| **PNG export**: `_compositeCanvas` / `exportCanvas` / `exportPNG` | 10042 / 10238 / 10292 | -| Native-resolution render `_withNativeSize` | 10018 | -| **Export UI**: `_toast` / `_downloadCanvas` / `_openMenu` | 10326 / 10420 / 10563 | -| Export registry `registerExportAction` | 10451 | +| Panel event dispatch `_attachPanelEvents` | 7787 | +| 2D events `_attachEvents2d` | 7829 | +| 1D events `_attachEvents1d` | 8222 | +| 2D widget drag `_ovHitTest2d` / `_doDrag2d` | 8497 / 8776 | +| **Brush strokes**: `_brushLiveBegin` / `_brushCommit` / `_brushErase` / `_brushPaintAt` | 8689 / 8703 / 8732 / 8767 | +| 1D widget drag `_canvasXToFrac1d` … / snapping `_snapVal` | 8901 / 8974 | +| Shared-axis propagation `_getShareGroups` | 9045 | +| Figure resize `_applyFigResizeDOM` | 9109 | +| **Bar chart**: `_barGeom` / `drawBar` / `_attachEventsBar` | 9300 / 9363 / 9739 | +| Generic redraw `_redrawPanel` | 9929 | +| **PNG export**: `_compositeCanvas` / `exportCanvas` / `exportPNG` | 10088 / 10284 / 10343 | +| Native-resolution render `_withNativeSize` | 10064 | +| **Export UI**: `_toast` / `_downloadCanvas` / `_openMenu` | 10377 / 10486 / 10665 | +| Export registry `registerExportAction` | 10542 | +| **Embedding API**: `createLocalModel` / `mount` | 11056 / 11112 | +| **Navigated embed**: `decodeBlocks` / `mountNavigated` | 11367 / 11754 | > **`brush` widget (2-D)** — the one widget whose drag is *modal*, and the one > that must NOT write the model per tick. `_ovHitTest2d` takes an extra `mods` @@ -157,7 +159,7 @@ geometry changes (visibility, label, sizes) re-layout automatically. ## Layout / panel details -#### `applyLayout()` (line 590) +#### `applyLayout()` (line 815) Reads `layout_json`. Builds CSS grid tracks from `panel_specs[].panel_width/height`. Creates panels that don't exist yet, resizes existing ones, removes stale ones. Also creates/updates inset panels from `inset_specs`, then draws region @@ -228,7 +230,7 @@ maps — otherwise `layout.indications` would keep emitting an entry whose `inset_id` no longer resolves to a live panel (caught by the `_drawCallouts` guard above, but a dangling entry all the same). -#### `_createPanelDOM(id, kind, pw, ph, spec)` (line 763) +#### `_createPanelDOM(id, kind, pw, ph, spec)` (line 1040) Builds all canvas/DOM elements for one panel (via `_buildCanvasStack`), stores the **`p` object** in `panels`, subscribes to `change:panel_{id}_json`, runs the initial draw. @@ -240,7 +242,7 @@ stores the **`p` object** in `panels`, subscribes to | `'3d'` | `wrap3 > plotCanvas + overlayCanvas + markersCanvas + statusBar` | | `'1d'` / `'bar'` | `wrap > plotCanvas + overlayCanvas + markersCanvas + statusBar` | -#### `_resizePanelDOM(id, pw, ph)` (line 1027) +#### `_resizePanelDOM(id, pw, ph)` (line 2288) Updates `canvas.width / canvas.height` (DPR-scaled) for every canvas in the panel. For 2D, computes `imgX/imgY/imgW/imgH` from the gutters (`PAD_*`, `_padT`, `_cbWidth`) and stores them on `p` plus `p._cbW`/`p._padT`. @@ -431,7 +433,7 @@ comparable to the base image's `_buildLut32` blit). --- -## 3D drawing (line ~1840) +## 3D drawing (line 5605) Orthographic projection; geometry b64-decoded and cached. `draw3d` sorts triangles, draws axes with per-axis `_drawTex` labels (`x/y/z_label_size`). @@ -569,14 +571,14 @@ triangles, draws axes with per-axis `_drawTex` labels (`x/y/z_label_size`). normalisation stays origin-true (unit-sphere direction vectors). ## Events -- `_emitEvent(panelId, eventType, widgetId, extraData)` (line 2031) writes +- `_emitEvent(panelId, eventType, widgetId, extraData)` (line 6442) writes `{source:'js', ...}` to `model.event_json`; `eventType` is any `pointer_*` / `key_*` / `wheel` / `double_click` string (see `callbacks.VALID_EVENT_TYPES`). - Kind-specific attach functions: 3D 2059, 2D 2928, 1D 3201, bar 4341. - Widget drag: 2D hit-test/drag 3409/3491; 1D from 3565. -## 1D drawing (line 2177) +## 1D drawing (line 6723) `draw1d` renders series (b64 decode cache), axes, ticks (log ticks as TeX `$10^{N}$`; edge labels nudged inward), grid, legend, units labels + title via `_drawTex` (title size clamped via `_titlePx`). @@ -636,13 +638,13 @@ exportCanvas(same opts) → {canvas, width, height} // synchronous, throws | Function | Line | Purpose | |----------|------|---------| -| `_cssScale` | 9918 | inverse of `_applyScale`'s `transform:scale()` | -| `_panelBox` | 9929 | the element whose rect bounds one panel | -| `_neutralizeView` / `_restoreView` | 9938 / 9963 | transient whole-extent view | -| `_nativeGeom` / `_nativeGuard` | 9978 / 9993 | native size + why-not message | -| `_withNativeSize` | 10018 | resize → redraw → run → restore | -| `_compositeCanvas` | 10042 | the compositor (`_drawEl` / `_drawPanel` …) | -| `exportCanvas` / `exportPNG` | 10238 / 10292 | orchestrator / data-URL wrapper | +| `_cssScale` | 9964 | inverse of `_applyScale`'s `transform:scale()` | +| `_panelBox` | 9975 | the element whose rect bounds one panel | +| `_neutralizeView` / `_restoreView` | 9984 / 10009 | transient whole-extent view | +| `_nativeGeom` / `_nativeGuard` | 10024 / 10039 | native size + why-not message | +| `_withNativeSize` | 10064 | resize → redraw → run → restore | +| `_compositeCanvas` | 10088 | the compositor (`_drawEl` / `_drawPanel` …) | +| `exportCanvas` / `exportPNG` | 10284 / 10343 | orchestrator / data-URL wrapper | **The whole pipeline is ONE synchronous task** — theme swap, view reset, native resize, composite, restore — so the browser never paints an intermediate state @@ -738,13 +740,13 @@ leaders that cross into the panel included. Pinned by | Function | Line | Purpose | |----------|------|---------| -| `_toast` | 10326 | transient bottom-centre message | -| `_copyCanvas` | 10361 | clipboard write + feature detection | -| `_showPngPreview` | 10385 | framed-document download fallback | -| `_downloadCanvas` | 10420 | `` or the preview | -| `registerExportAction` | 10451 | downstream extension point | -| `_menuRows` / `_openMenu` | 10504 / 10563 | menu model / DOM | -| `_panelAtPoint` | 10673 | hit test (insets first — they sit on top) | +| `_toast` | 10377 | transient bottom-centre message | +| `_copyCanvas` | 10412 | clipboard write + feature detection | +| `_showPngPreview` | 10436 | framed-document download fallback | +| `_downloadCanvas` | 10486 | `` or the preview | +| `registerExportAction` | 10542 | downstream extension point | +| `_menuRows` / `_openMenu` | 10596 / 10665 | menu model / DOM | +| `_panelAtPoint` | 10777 | hit test (insets first — they sit on top) | - **An `exportBtn` badge (⤓, beside the help badge) opens the same menu on an ordinary left click.** It is a `role="button"` with `tabIndex=0` and @@ -815,3 +817,159 @@ render()'s api into `_aplRenderApi`, **also assigns it to `window._aplRenderApi` `{type:'anyplotlib_export_png_result', requestId, dataUrl, width, height}` (or `{…, error}`) to `event.source` (targetOrigin `'*'`). `opts` is forwarded verbatim, so the new fields work over that channel too. + +--- + +## Navigated-embed runtime (line 11333 to the end of the file) + +Everything below `mount()` is module scope, outside `render()`'s closure: pure +functions over decoded data plus one entry point that wires them to a mounted +figure. A navigated page — a navigator panel whose widget drives a signal +panel and its overlays — is then "mount the figure, hand it blocks and +bindings, let it dispatch", rather than a hand-written program per result kind. + +| Function | Line | Purpose | +|----------|------|---------| +| `decodeBlocks` | 11367 | one base64 `fetch` → one ArrayBuffer → a typed-array view per manifest entry | +| `dense` | 11393 | `at` / `gather` / `reduce` over a block whose leading axes are the nav axes | +| `ragged` | 11458 | the same three, over a row-pointer block (`offsets` + one array per column) | +| `maskFromWidget` | 11541 | rectangle / circle / annulus widget dict → `Uint8Array` (carries `width`/`height`) | +| `rasterDisks` | 11581 | splat `{x, y, intensity}` rows as filled disks — the base image of a vectors panel | +| `robustLevels` / `toU8` | 11610 / 11651 | the percentile window and the 8-bit code map, one implementation | +| `panelAxis` | 11729 | a 1-D panel's decoded x axis (`_1dXArr`, else `x_axis_b64`) | +| `installTouchShim` / `reportEmbedHeight` | 11666 / 11685 | page chrome: touch → mouse, `postMessage({aplEmbedHeight})` | +| `encodeBase64` / `typedArrayBytes` | 11705 / 11713 | a 3-D cloud's geometry channel is base64, not the binary side table | +| `mountNavigated` | 11754 | mount + bind + dispatch; resolves to the mount handle plus `dispatch`/`index`/`blocks` | + +`mountNavigated(el, page, opts)` is **async** — the blob decode is a `fetch` of +a `data:` URL — so a host `await`s it. `page` is `{state, blocks, bindings, +chrome}` as `anyplotlib.embed.navigated_html` inlines it. The generic readers +are reachable as `embed.dense`, `embed.ragged` and so on rather than as +top-level exports, so `dense` and `toU8` do not sit beside `mount` and `render` +in an importer's completion list. + +**Dispatch.** A navigator widget's `pointer_move` / `pointer_up` maps to a +navigation index (crosshair: rounded `cy, cx`; rectangle: the index set, capped +by the widget's own `max_w`/`max_h`; vline: the nearest `x_axis` entry) and every +`role: "driven"` binding is refreshed — frame through `at`/`gather` → `toU8` → +`setImage`, overlays through `patchPanel({markers})` (2-D) or +`patchPanel({extra_lines})` (1-D), readout and chips into their strip elements. +Event-driven dispatches coalesce on one `requestAnimationFrame`, latest index +wins; `handle.dispatch(index)` is synchronous so a caller (or a test) can drive +it directly. A detector widget on a driven panel whose binding carries +`reduce` runs the other way: `maskFromWidget` → `reduce` → the NAVIGATOR's +image. + +**The navigator panel IS the navigation grid**, which is why no binding has to +declare its shape: a 2-D widget reports `cx`/`cy` in image pixels, so those +already are the index. A **1-D** navigator is the other half of that: `vline` +and `point` resolve their data coordinate through the panel's own x axis, and +`range` (the span selector, the 1-D analogue of the rectangle) resolves both +edges and selects the run between them. That axis travels base64-encoded as +`x_axis_b64`, NOT as `x_axis` — `Plot1D.to_state_dict` pops the plain key — so +`panelAxis` reads `draw1d`'s decoded cache (`p._1dXArr`) and decodes only as a +fallback. Reading `state.x_axis` gives an empty array and every position +resolves to 0. + +**A dispatch owns only the groups it wrote.** Marker and extra-line groups the +runtime creates carry the `apl-overlay-` id prefix, and `paintOverlays` merges +by that prefix rather than assigning the list — otherwise the first crosshair +move wipes the annotations the figure was built with. + +**An overlay's `style` IS its wire dict** (the keys `MarkerGroup.to_wire` +emits), carried through whole rather than read key by key: an allow-list drops +whatever it has not heard of, and a `fill_alpha` of 0 or 1 reads as "unset" to +a `style.fill_alpha || default` test and silently becomes the renderer's 0.3. + +**A 3-D panel is driven through `patchPanel`, never `setImage`.** Its cloud +(`vertices_b64`, `point_colors_b64`, `z_values_b64`) rides `panel__geom` as +BASE64: the binary side table is registered for image pixel keys only, and a +cloud changes on a view click rather than per navigator move, so the encode is +not on a hot path. `vertices_count` and a bumped `_geom_rev` go on the light +trait in the same breath, or the renderer draws the previous cloud's point +count. `paintPoints3d` skips when the (block, colours) pair is already shown, +so a navigator drag costs nothing. + +A `"highlight"` overlay writes the `{x, y, z, color, size}` shape +`Plot3D.set_highlight` writes. With `face_camera` it also writes the camera: +`elevation = asin(z/r)`, `azimuth = atan2(x, -y)` in degrees, plus +`_view_from_python: true` so `_preserveView` lets the intended camera through. +The flag is written EITHER WAY, because it persists on the trait: a `true` left +by a previous facing push would let the next highlight discard the orbit the +reader is holding. + +**A `views` binding is a committed result's alternative frames** (a strain +map's εxx / εyy / εxy / ω): the page renders a segmented control, and picking one +swaps which block `frameBlock` reads at whatever position the navigator is +already on. It is not a per-position scalar — `readout` covers those. An +entry is identified by its POSITION in the list, not by its block: two views +may read one block with different colours, which is what a direction toggle +over a single point cloud looks like. + +**The frame's colour window is the panel's**, not a fresh percentile window per +frame: recomputing costs two passes over the data on every dispatch AND makes +the contrast jump between neighbouring positions, which reads as the data +changing. `robustLevels` is the fallback for a panel with no window. + +**A detector is a rectangle, a circle or an annulus.** `mountNavigated` refuses +a `reduce` binding whose panel carries none of those, because the binding could +otherwise never fire and nothing would say so; at dispatch, any other widget on +that panel is simply not a detector (`maskFromWidget` would throw inside an +animation frame, where nothing catches it). + +## `setImage` / `patchPanel` / `panelIds` on the mount handle + +| Handle method | What it writes | +|---------------|----------------| +| `setImage(panelId, bytes, w, h, opts)` | queues the frame; the next animation frame writes `globalThis.__apl_pixbytes["panel__geom::image_b64"]`, a fresh `\u0000bin:` token in the panel's `_geomCache`, the geometry patch, and one `applyRemote` on that slot | +| `patchPanel(panelId, partial)` | parse `panel__json`, `Object.assign`, `applyRemote` — values verbatim | +| `panelIds()` | `layout_json.panel_specs[].id`, in layout order | +| `flushImages()` | paint pending `setImage` frames now instead of on the next frame | + +`setImage` exists because the geom trait is the wrong channel for a scrub: +measured on 0.8.0, a frame pushed as base64 costs 6.7 ms at 512² and 129-136 ms +at 2048² of main-thread time, against 0.3 ms either way for the raw bytes. + +Two things about it are load-bearing and look like clutter: + +- **The `_geomCache.image_b64` token.** `_imageBytes` keys its blit cache on + that string first (arrival sequence is only its fallback), and a `mount()` + page's geom carries REAL base64 that never changes. Without a fresh token + per push the cache reports "unchanged" and the new pixels never reach the + canvas. This is the same trick `_electron._route_change` plays with an + adler32 content token. +- **The repaint is deferred to `requestAnimationFrame`.** The push itself is a + side-table write; the LUT blit it schedules is 15-25 ms at 2048² and would + otherwise land on the caller's thread, one blit per pushed frame. Coalescing + per panel means a task that pushes several frames paints the last one once. + `exportPNG` / `exportCanvas` call `flushImages()` first, so an export never + captures the frame before the one just handed over. +- **The geometry patch rides the SAME animation frame as the bytes** (`imagePatch`, + applied inside `commitImages`). Patching at push time instead lets one frame + paint at the new `image_width`/`image_height` over the previous pixels. +- **The sequence counter is `globalThis.__apl_pixseq`, not per handle.** The + side table is global and panel ids hash the layout position, so two + identical-layout figures in one document would otherwise mint the same key. +- **`setImage` refuses anything but a 2-D panel.** A 3-D or 1-D panel has a + geometry trait too, so without the check the bytes are accepted and go + nowhere. +- **A new frame voids the detail tile.** A tile is a crop of the PREVIOUS frame + at a zoom the viewer may still be sitting at, and `_blit2d` composites it over + the base, so the light `detail_*` fields, the geom cache's `detail_b64` / + `detail_b64_bytes`, the side-table slot and `p._detailBlit` all go with the + frame they came from. +- **`_loadGeom` keeps a live `\u0000bin:` token** when the cache holds + `image_b64_bytes` and the incoming geom does not carry a token of its own: the + token names the bytes that are actually drawn (`_imageBytes` prefers the + bytes), so letting a geom push rename it to stale base64 desynchronises the + blit cache key from its contents. A push WITH its own token (the Electron + `_route_change` path) is newer and wins. + +`opts.rgb` means **RGBA, four bytes per pixel** — the renderer's `is_rgb` path +sets `ImageData` straight from the bytes, so three-byte rows would be read as +RGBA and shear. + +Tests: `tests/test_embed/test_embed_set_image.py` (push cost at 512²/2048², +painting, the throws, no `onSync` echo), `test_embed_navigated.py` (the page +end to end: drag, overlay, detector reduce, region mean, the readers against +numpy), `test_embed_api.py` (`pack_blocks` round trip, `navigated_html`). diff --git a/anyplotlib/_repr_utils.py b/anyplotlib/_repr_utils.py index a7b4e099..0c3d79ab 100644 --- a/anyplotlib/_repr_utils.py +++ b/anyplotlib/_repr_utils.py @@ -34,6 +34,23 @@ # Trait serialisation # --------------------------------------------------------------------------- +def script_json(obj) -> str: + """Return *obj* as a JSON literal that is safe inside a ```` would close the + block early and run whatever followed it as markup. Both sequences are + escaped through the ``<``, which JSON spells ``\u003c``, so ``JSON.parse`` + and a script literal read back exactly the character that went in. + """ + return (json.dumps(obj, default=str) + .replace(" dict: """Return a {name: value} dict of every synced traitlet. @@ -249,6 +266,7 @@ def _widget_px(widget) -> tuple[int, int]: // host page script cannot reach exportPNG (only the postMessage protocol // below can). anyplotlib.savefig() drives the export through this handle. window._aplRenderApi = _aplRenderApi; + globalThis.__aplExportPNG = (o) => _aplRenderApi.exportPNG(o); }} else {{ el.textContent = "ESM has no render() export"; }} @@ -304,45 +322,55 @@ def _widget_px(widget) -> tuple[int, int]: }} }}); +{png_harvest} + + + +""" + + +# A host page (or the SpyDE report harvester) asks an embedded figure for a +# composite PNG over postMessage. Both the standalone page and the navigated +# embed install this listener; each assigns ``globalThis.__aplExportPNG`` once +# its figure is mounted, which is also what makes "not ready yet" answerable. +PNG_HARVEST_LISTENER = '''\ // ── PNG export protocol ────────────────────────────────────────────────────── -// Rides the same postMessage channel as the state updates above. A parent page -// (or the SpyDE report harvester) requests a composite PNG of the whole figure: -// → {{ type: 'anyplotlib_export_png', requestId, opts }} +// A parent page (or the SpyDE report harvester) asks this frame over +// postMessage for a composite PNG of the whole figure: +// → { type: 'anyplotlib_export_png', requestId, opts } // and receives back, on event.source (targetOrigin '*'): -// ← {{ type: 'anyplotlib_export_png_result', requestId, dataUrl, width, height }} -// ← {{ type: 'anyplotlib_export_png_result', requestId, error }} (on failure) +// ← { type: 'anyplotlib_export_png_result', requestId, dataUrl, width, height } +// ← { type: 'anyplotlib_export_png_result', requestId, error } (on failure) // `opts` is forwarded verbatim to exportPNG: -// {{ scale?, includeWidgets?, panelId?, source?, theme? }} +// { scale?, includeWidgets?, panelId?, source?, theme? } // source: 'view' | 'full' | 'native' theme: 'current' | 'light' | 'dark' -window.addEventListener('message', (e) => {{ +window.addEventListener('message', (e) => { if (!e.data || e.data.type !== 'anyplotlib_export_png') return; const requestId = e.data.requestId; const source = e.source; - const reply = (msg) => {{ - try {{ - if (source && typeof source.postMessage === 'function') {{ + const reply = (msg) => { + try { + if (source && typeof source.postMessage === 'function') { source.postMessage(Object.assign( - {{ type: 'anyplotlib_export_png_result', requestId }}, msg), '*'); - }} - }} catch (_) {{}} - }}; - try {{ - if (!_aplRenderApi || typeof _aplRenderApi.exportPNG !== 'function') {{ - reply({{ error: 'figure not ready (exportPNG unavailable)' }}); + { type: 'anyplotlib_export_png_result', requestId }, msg), '*'); + } + } catch (_) {} + }; + try { + const exportPNG = globalThis.__aplExportPNG; + if (typeof exportPNG !== 'function') { + reply({ error: 'figure not ready (exportPNG unavailable)' }); return; - }} - Promise.resolve(_aplRenderApi.exportPNG(e.data.opts || {{}})) - .then((res) => reply({{ - dataUrl: res.dataUrl, width: res.width, height: res.height }})) - .catch((err) => reply({{ error: String(err && err.message || err) }})); - }} catch (err) {{ - reply({{ error: String(err && err.message || err) }}); - }} -}}); - - - -""" + } + Promise.resolve(exportPNG(e.data.opts || {})) + .then((res) => reply({ + dataUrl: res.dataUrl, width: res.width, height: res.height })) + .catch((err) => reply({ error: String(err && err.message || err) })); + } catch (err) { + reply({ error: String(err && err.message || err) }); + } +}); +''' def build_standalone_html(widget, *, resizable: bool = True, @@ -375,9 +403,10 @@ def build_standalone_html(widget, *, resizable: bool = True, width=w, height=h, extra_css=extra_css, - state_json=json.dumps(state, default=str), - esm_json=json.dumps(esm), - fig_id_json=json.dumps(fig_id), + state_json=script_json(state), + esm_json=script_json(esm), + fig_id_json=script_json(fig_id), + png_harvest=PNG_HARVEST_LISTENER, ) diff --git a/anyplotlib/embed.py b/anyplotlib/embed.py index 6b17d317..56fe086d 100644 --- a/anyplotlib/embed.py +++ b/anyplotlib/embed.py @@ -55,11 +55,19 @@ from __future__ import annotations +import base64 +import dataclasses import pathlib +from html import escape -from anyplotlib._repr_utils import build_standalone_html, _widget_state +import numpy as np -__all__ = ["figure_state", "to_html", "save_html", "esm_path", "FigureBridge"] +from anyplotlib._repr_utils import ( + PNG_HARVEST_LISTENER, build_standalone_html, script_json, _widget_state, +) + +__all__ = ["figure_state", "to_html", "save_html", "esm_path", "FigureBridge", + "Ragged", "pack_blocks", "navigated_html"] def figure_state(fig) -> dict: @@ -192,3 +200,258 @@ def close(self) -> None: self._fig.unobserve(self._on_trait_change, names=traitlets.All) except ValueError: pass + + +# --------------------------------------------------------------------------- +# Navigated pages +# --------------------------------------------------------------------------- + +#: The dtypes the JS runtime has a typed array for. Anything else has to be +#: cast before packing, because a view it cannot name is a silent wrong answer. +_BLOCK_DTYPES = frozenset({"uint8", "int8", "uint16", "int16", + "uint32", "int32", "float32", "float64"}) + +#: Block offsets are padded to this, the largest element size above. +_BLOCK_ALIGNMENT = 8 + + +@dataclasses.dataclass +class Ragged: + """A block with a variable number of rows per navigation position. + + ``offsets`` is the row-pointer array: position ``i`` owns rows + ``offsets[i]`` up to ``offsets[i + 1]``, so it has ``n_positions + 1`` + entries. ``columns`` maps a name to one value per row. ``nav_shape`` + gives the navigation grid when it has more than one axis, so a + two-dimensional index resolves to the right row span. + """ + + offsets: np.ndarray + columns: dict + nav_shape: tuple = () + + +def _block_dtype_name(array) -> str: + name = str(array.dtype) + if name not in _BLOCK_DTYPES: + raise ValueError( + f"block dtype {name!r} cannot be viewed by the page; cast it to one " + f"of {', '.join(sorted(_BLOCK_DTYPES))} first") + return name + + +def pack_blocks(blocks: dict) -> tuple[bytes, dict]: + """Pack arrays into one little-endian byte string plus a manifest. + + *blocks* maps a name to a numpy array (a dense block whose leading axes are + the navigation axes) or to a :class:`Ragged`. The return is + ``(payload, manifest)``: the page base64-decodes *payload* once into a + single ``ArrayBuffer`` and takes a typed-array view per manifest entry, so + no block is encoded or copied on its own. + """ + payload = bytearray() + manifest: dict = {} + + def append(array) -> dict: + contiguous = np.ascontiguousarray(array) + dtype_name = _block_dtype_name(contiguous) + # A typed array can only view an offset that is a multiple of its + # element size. + padding = (-len(payload)) % _BLOCK_ALIGNMENT + payload.extend(b"\0" * padding) + spec = {"dtype": dtype_name, "offset": len(payload), + "nbytes": int(contiguous.nbytes)} + payload.extend(contiguous.astype(contiguous.dtype.newbyteorder("<"), + copy=False).tobytes()) + return spec + + for name, block in blocks.items(): + if isinstance(block, Ragged): + offsets = np.ascontiguousarray(block.offsets, dtype=np.int32) + nav_shape = tuple(block.nav_shape) or (int(offsets.size) - 1,) + entry = {"kind": "ragged", "nav_shape": [int(n) for n in nav_shape], + "offsets": append(offsets), + "columns": {column: append(values) + for column, values in block.columns.items()}} + else: + array = np.ascontiguousarray(block) + entry = dict(append(array), kind="dense", + shape=[int(n) for n in array.shape]) + manifest[name] = entry + + return bytes(payload), manifest + + +def _validate_bindings(state: dict, blocks: dict, bindings: list) -> None: + """Raise when a binding names a panel or a block the page does not have.""" + panel_ids = {key[len("panel_"):-len("_json")] for key in state + if key.startswith("panel_") and key.endswith("_json")} + for binding in bindings: + panel_id = binding.get("panel_id") + if panel_id not in panel_ids: + raise ValueError(f"binding names unknown panel {panel_id!r}; " + f"the figure has {sorted(panel_ids)}") + names = [] + frame = binding.get("frame") or {} + if frame.get("block"): + names.append(frame["block"]) + if frame.get("colors"): + names.append(frame["colors"]) + for overlay in binding.get("overlays") or []: + names.append(overlay["block"]) + if binding.get("reduce"): + names.append(binding["reduce"]["block"]) + navigator = binding["reduce"]["navigator_panel"] + if navigator not in panel_ids: + raise ValueError(f"reduce names unknown navigator panel {navigator!r}") + for view in binding.get("views") or []: + names.append(view["block"]) + if view.get("colors"): + names.append(view["colors"]) + if binding.get("readout"): + names.append(binding["readout"]["block"]) + for name in names: + if name not in blocks: + raise ValueError(f"binding names unknown block {name!r}; " + f"the page carries {sorted(blocks)}") + + +def _views_control(binding: dict) -> str: + """The segmented control that picks which block a panel's frame comes from.""" + # The button names its POSITION, not its block: two views may read the + # same block with different colours (a direction toggle on one cloud). + buttons = "".join( + f'' + for index, view in enumerate(binding["views"])) + panel_id = escape(str(binding["panel_id"]), quote=True) + return f'
{buttons}
' + + +_NAVIGATED_PAGE = """\ + + + + + +{title} + + + +
+{title_html}
{strips_html}{caption_html} +
+ + + +""" + + +def navigated_html(fig_or_state, blocks: dict, bindings: list, *, + chrome: dict | None = None, title: str = "", + caption: str = "") -> str: + """Return a self-contained page whose navigator drives its other panels. + + *fig_or_state* is a live ``Figure`` or the dict :func:`figure_state` + returns. *blocks* is the data the page navigates, in the form + :func:`pack_blocks` takes. *bindings* says what each panel does:: + + {panel_id, role: "navigator" | "driven" | "static", + widgets: [...], + frame: {block, kind: "image" | "disks" | "points3d", radius?, + combine?, levels?, width?, height?, colors?}, + views: [{label, block, colors?}], + overlays: [{block, kind, style, columns?, face_camera?}], + reduce: {block, navigator_panel, x?, y?, value?}, + readout: {block, names, units}} + + ``views`` are a committed result's alternative frames (a strain map's + epsilon_xx, epsilon_yy, epsilon_xy, omega): the page renders a segmented + control that swaps which block the panel's frame is read from, at whatever + position the navigator is already on. With ``views``, ``frame.block`` may + be omitted and the first entry is the one shown first. A navigator binding + may carry ``initial_index`` to open somewhere other than the origin. + + A 3-D panel takes ``kind: "points3d"`` (a dense ``(M, 3)`` float32 cloud + plus a ``colors`` block) and a ``"highlight"`` overlay marking one point per + navigation position; ``face_camera`` on that overlay turns the panel to face + the marked point. + + The renderer, the figure state, the packed data and the bindings are all + inlined, so the page needs no network and no Python at view time. + + Raises + ------ + ValueError + When a binding names a panel or a block the page does not carry. + """ + state = (fig_or_state if isinstance(fig_or_state, dict) + else figure_state(fig_or_state)) + payload, manifest = pack_blocks(blocks) + _validate_bindings(state, manifest, bindings) + + page = {"state": state, "bindings": bindings, "chrome": chrome or {}, + "blocks": {"data": base64.b64encode(payload).decode("ascii"), + "manifest": manifest}} + + strips = "".join( + '
' + for binding in bindings if binding.get("readout")) + strips += "".join(_views_control(binding) + for binding in bindings if binding.get("views")) + + return _NAVIGATED_PAGE.format( + title=escape(title or "anyplotlib figure"), + title_html=(f'
{escape(title)}
\n' if title else ""), + caption_html=(f'
{escape(caption)}
' + if caption else ""), + strips_html=strips, + page_json=script_json(page), + esm_json=script_json(esm_path().read_text(encoding="utf-8")), + png_harvest=PNG_HARVEST_LISTENER, + ) diff --git a/anyplotlib/figure_esm.js b/anyplotlib/figure_esm.js index 12aabd57..30d6bc68 100644 --- a/anyplotlib/figure_esm.js +++ b/anyplotlib/figure_esm.js @@ -416,6 +416,15 @@ function render({ model, el, onResize, onReadout }) { for (const k in prev) { if (k.endsWith('_bytes') && next[k] === undefined) next[k] = prev[k]; } + // A live binary frame (setImage, or a PLOTBIN push) is drawn from + // `image_b64_bytes`, and the token in `image_b64` is the cache key for + // exactly those bytes. A geom push carrying base64 would replace the key + // while the bytes stay, freezing the display on whatever was last drawn. + // A push with its own token is newer and wins. + if (typeof prev.image_b64 === 'string' && prev.image_b64.startsWith('\u0000bin:') + && next.image_b64_bytes + && !(typeof next.image_b64 === 'string' && next.image_b64.startsWith('\u0000bin:'))) + next.image_b64 = prev.image_b64; p2._geomCache = next; p2._geomRev = rev; } catch (_) {} @@ -11120,7 +11129,84 @@ export function mount(el, state, opts) { // embedding host can relayout the figure to its new box. const api = render({ model, el, onResize: o.onResize, onReadout: o.onReadout }) || {}; - return { + + // Raw-pixel pushes (setImage) coalesce onto one animation frame per panel: + // several frames handed over inside one task repaint once, showing the last. + // Without this a scrub pays the full LUT blit per frame on the caller's + // thread, which is the cost the binary path exists to avoid. + const pendingImages = new Map(); // panel id → the frame to paint next + let imageFrameRequest = null; + + function commitImages() { + imageFrameRequest = null; + const frames = [...pendingImages]; + pendingImages.clear(); + for (const [panelId, frame] of frames) { + const panel = api.panels && api.panels.get(panelId); + if (!panel) continue; + // Geometry is patched HERE rather than at push time so it lands in the + // same task as the bytes it describes: a frame of a new size would + // otherwise paint once at the new dimensions over the previous bytes. + const patch = imagePatch(panel.state || {}, frame); + if (Object.keys(patch).length) handle.patchPanel(panelId, patch); + const slot = `panel_${panelId}_geom::image_b64`; + const table = globalThis.__apl_pixbytes || (globalThis.__apl_pixbytes = {}); + table[slot] = frame.bytes; + // The blit cache keys on the geom's image_b64 string, so a frame pushed + // under an unchanged key would never reach the canvas. The counter is + // per DOCUMENT, not per handle: the side table is global and panel ids + // repeat across figures of the same layout. + const sequence = (globalThis.__apl_pixseq = (globalThis.__apl_pixseq || 0) + 1); + if (!panel._geomCache) panel._geomCache = {}; + panel._geomCache.image_b64 = `\u0000bin:${sequence}`; + panel._geomCache.detail_b64 = ''; + delete panel._geomCache.detail_b64_bytes; + delete panel._detailBlit; + delete globalThis.__apl_pixbytes[`panel_${panelId}_geom::detail_b64`]; + model.applyRemote(slot, `${frame.bytes.length}:${sequence}`); + } + } + + // The state fields a pushed frame implies, limited to the ones that differ. + function imagePatch(current, frame) { + const patch = {}; + if (current.image_width !== frame.width) patch.image_width = frame.width; + if (current.image_height !== frame.height) patch.image_height = frame.height; + if (!!current.is_rgb !== frame.rgb) patch.is_rgb = frame.rgb; + // These bytes are the whole frame, so any overview/tile geometry the panel + // was built with no longer describes them. + if (current.base_width) patch.base_width = 0; + if (current.base_height) patch.base_height = 0; + if (current.tile_enabled) patch.tile_enabled = false; + // A detail tile is a crop of the PREVIOUS frame at a zoom the viewer may + // still be at, and _blit2d composites it over the base. Clearing only the + // light field leaves the bytes and the region in the geom cache, which + // _applyGeom splices straight back in. + if (current.detail_b64 || current.detail_width || current.detail_height) + Object.assign(patch, { detail_b64: '', detail_region: [], + detail_width: 0, detail_height: 0, + detail_min: null, detail_max: null, + detail_is_int: false }); + // display_* is the colour window; raw_* is the band the codes span. The + // caller hands over codes already mapped to its window, so they agree. + if (frame.displayMin !== undefined && current.display_min !== frame.displayMin) { + patch.display_min = frame.displayMin; + patch.raw_min = frame.displayMin; + } + if (frame.displayMax !== undefined && current.display_max !== frame.displayMax) { + patch.display_max = frame.displayMax; + patch.raw_max = frame.displayMax; + } + return patch; + } + + function flushImages() { + if (imageFrameRequest !== null) cancelAnimationFrame(imageFrameRequest); + if (pendingImages.size) commitImages(); + imageFrameRequest = null; + } + + const handle = { model, api, // internal render() API (panels, calloutCanvas, _drawCallouts, …) get(key) { return model.get(key); }, @@ -11130,6 +11216,56 @@ export function mount(el, state, opts) { const v = typeof panelState === 'string' ? panelState : JSON.stringify(panelState); this.set('panel_' + panelId + '_json', v); }, + // Merge *partial* into one panel's state and re-render it. Values are + // stored verbatim, so this is how markers, extra_lines, display_min / + // display_max and overlay_widgets are driven from JS. + patchPanel(panelId, partial) { + const key = `panel_${panelId}_json`; + if (model.get(key) === undefined) + throw new Error(`patchPanel: unknown panel id ${panelId}`); + let current = {}; + try { current = JSON.parse(model.get(key) || '{}'); } catch (_) {} + model.applyRemote(key, JSON.stringify(Object.assign(current, partial))); + }, + // The panel ids in layout order, so a host need not parse layout_json. + panelIds() { + try { + const layout = JSON.parse(model.get('layout_json') || '{}'); + return (layout.panel_specs || []).map((spec) => spec.id); + } catch (_) { return []; } + }, + // Replace a 2-D panel's image with RAW pixel bytes, skipping base64. + // bytes Uint8Array of width*height colormap codes, or width*height*4 + // RGBA bytes when opts.rgb is true. + // opts {rgb, display_min, display_max}. Each is patched into the + // panel's state first when it differs, so the geometry and the + // colour window match the bytes on the very frame they arrive. + // Throws on an unknown panel or a byte count that is not width*height. + // The repaint lands on the next animation frame; exportPNG flushes first. + setImage(panelId, bytes, width, height, opts) { + const panel = api.panels && api.panels.get(panelId); + if (!panel) throw new Error(`setImage: unknown panel id ${panelId}`); + if (panel.kind !== '2d') + throw new Error(`setImage: panel ${panelId} is a ${panel.kind} panel; ` + + `only a 2-D image panel draws pixel bytes`); + if (model.get(`panel_${panelId}_geom`) === undefined) + throw new Error(`setImage: panel ${panelId} has no image channel`); + const settings = opts || {}; + const rgb = !!settings.rgb; + const expected = width * height * (rgb ? 4 : 1); + if (!bytes || bytes.length !== expected) + throw new Error(`setImage: expected ${expected} bytes for ${width}x${height}` + + `${rgb ? ' RGBA' : ''}, got ${bytes ? bytes.length : 0}`); + + pendingImages.set(panelId, { + bytes, width, height, rgb, + displayMin: settings.display_min, displayMax: settings.display_max, + }); + if (imageFrameRequest === null) + imageFrameRequest = requestAnimationFrame(commitImages); + }, + // Paint every pending setImage frame now instead of on the next frame. + flushImages, // Inbound update from a Python bridge — renders without echoing to onSync. applyUpdate(key, value) { model.applyRemote(key, value); }, resize(width, height) { @@ -11148,7 +11284,7 @@ export function mount(el, state, opts) { if (typeof api.exportPNG !== 'function') { return Promise.reject(new Error('exportPNG unavailable (render() returned no API)')); } - try { return api.exportPNG(opts); } + try { flushImages(); return api.exportPNG(opts); } catch (e) { return Promise.reject(e); } }, // Same, but synchronous and returning the raw {canvas, width, height} so a @@ -11157,6 +11293,7 @@ export function mount(el, state, opts) { exportCanvas(opts) { if (typeof api.exportCanvas !== 'function') throw new Error('exportCanvas unavailable (render() returned no API)'); + flushImages(); return api.exportCanvas(opts); }, // Add an entry to the right-click export menu. Returns an unregister fn. @@ -11184,10 +11321,891 @@ export function mount(el, state, opts) { model.off(); el.replaceChildren(); }, }; + return handle; +} + + + + + + + +// ═══════════════════════════════════════════════════════════════════════════ +// Navigated-embed runtime: a page that owns its data and dispatches on it. +// +// A navigated page is a navigator panel whose widget drives one or more other +// panels: move the crosshair, the signal panel shows that position's frame and +// its overlays follow. The page carries the whole dataset as one binary blob +// plus a list of bindings that say which block feeds which panel, so the +// dispatch is a table lookup and a `setImage`, not a program per result kind. +// +// import { mountNavigated } from './figure_esm.js'; +// const handle = await mountNavigated(host, page); // page from Python +// handle.dispatch([3, 5]); // navigate from code +// +// `page` is what `anyplotlib.embed.navigated_html` inlines: +// {state, blocks: {data, manifest}, bindings: [...], chrome: {...}} +// ═══════════════════════════════════════════════════════════════════════════ + +const BLOCK_ARRAY_TYPES = { + uint8: Uint8Array, int8: Int8Array, + uint16: Uint16Array, int16: Int16Array, + uint32: Uint32Array, int32: Int32Array, + float32: Float32Array, float64: Float64Array, +}; + +function blockArrayView(buffer, spec) { + const ArrayType = BLOCK_ARRAY_TYPES[spec.dtype]; + if (!ArrayType) throw new Error(`unsupported block dtype ${spec.dtype}`); + return new ArrayType(buffer, spec.offset, spec.nbytes / ArrayType.BYTES_PER_ELEMENT); +} + +// Decode one packed blob into typed-array views over a single ArrayBuffer. +// `packed` is {data: base64 string, manifest: {name: entry}} as produced by +// `anyplotlib.embed.pack_blocks`; every view aliases the same buffer, so +// nothing is copied per block. +async function decodeBlocks(packed) { + const manifest = (packed && packed.manifest) || {}; + const payload = (packed && packed.data) || ''; + const buffer = payload + ? await (await fetch(`data:application/octet-stream;base64,${payload}`)).arrayBuffer() + : new ArrayBuffer(0); + const blocks = {}; + for (const name of Object.keys(manifest)) { + const entry = manifest[name]; + if (entry.kind === 'ragged') { + const columns = {}; + for (const column of Object.keys(entry.columns)) + columns[column] = blockArrayView(buffer, entry.columns[column]); + blocks[name] = { kind: 'ragged', columns, + offsets: blockArrayView(buffer, entry.offsets), + navShape: entry.nav_shape || null }; + } else { + blocks[name] = { kind: 'dense', shape: entry.shape, + array: blockArrayView(buffer, entry) }; + } + } + return blocks; +} + +// Reader over a dense block whose LEADING dimensions are the navigation axes. +// `at` returns a view (never a copy); `gather` and `reduce` return new arrays. +function dense(block) { + const shape = block.shape; + const array = block.array; + + function frameLength(navigationDimensions) { + let length = 1; + for (let axis = navigationDimensions; axis < shape.length; axis++) length *= shape[axis]; + return length; + } + + function flatPosition(index) { + const parts = Array.isArray(index) ? index : [index]; + let flat = 0; + for (let axis = 0; axis < parts.length; axis++) { + const extent = shape[axis]; + const clamped = Math.max(0, Math.min(extent - 1, Math.round(parts[axis]))); + flat = flat * extent + clamped; + } + return flat; + } + + return { + kind: 'dense', + shape, + at(index) { + const parts = Array.isArray(index) ? index : [index]; + const length = frameLength(parts.length); + const start = flatPosition(parts) * length; + return array.subarray(start, start + length); + }, + gather(indices) { + if (!indices.length) return new Float32Array(0); + const total = new Float32Array(this.at(indices[0]).length); + for (const index of indices) { + const frame = this.at(index); + for (let i = 0; i < total.length; i++) total[i] += frame[i]; + } + const scale = 1 / indices.length; + for (let i = 0; i < total.length; i++) total[i] *= scale; + return total; + }, + // One value per navigation position: sum(frame * mask) over the signal + // grid. `mask` is a Uint8Array covering the trailing two dimensions. + reduce(mask) { + const signalLength = shape[shape.length - 2] * shape[shape.length - 1]; + if (mask.length !== signalLength) + throw new Error(`reduce: mask of ${mask.length} does not cover the ` + + `${shape[shape.length - 2]}x${shape[shape.length - 1]} signal grid`); + const selected = []; + for (let i = 0; i < signalLength; i++) if (mask[i]) selected.push(i); + const positions = Math.floor(array.length / signalLength); + const out = new Float32Array(positions); + for (let position = 0; position < positions; position++) { + const base = position * signalLength; + let total = 0; + for (let i = 0; i < selected.length; i++) total += array[base + selected[i]]; + out[position] = total; + } + return out; + }, + }; +} + +// Reader over a ragged block: a row-pointer array plus one value array per +// column, so each navigation position owns a variable number of rows. +function ragged(block) { + const offsets = block.offsets; + const columns = block.columns; + const columnNames = Object.keys(columns); + const navShape = block.navShape || [offsets.length - 1]; + + function flatPosition(index) { + const parts = Array.isArray(index) ? index : [index]; + if (parts.length !== navShape.length) + throw new Error(`ragged index of length ${parts.length} into a ` + + `${navShape.length}-D navigation grid; pack the block ` + + `with nav_shape if it has more than one axis`); + let flat = 0; + for (let axis = 0; axis < navShape.length; axis++) { + const extent = navShape[axis]; + const clamped = Math.max(0, Math.min(extent - 1, Math.round(parts[axis]))); + flat = flat * extent + clamped; + } + return flat; + } + + return { + kind: 'ragged', + navShape, + columnNames, + at(index) { + const position = flatPosition(index); + const start = offsets[position], stop = offsets[position + 1]; + const rows = {}; + for (const name of columnNames) rows[name] = columns[name].subarray(start, stop); + return rows; + }, + gather(indices) { + const spans = []; + let total = 0; + for (const index of indices) { + const position = flatPosition(index); + const start = offsets[position], stop = offsets[position + 1]; + spans.push([start, stop]); + total += stop - start; + } + const rows = {}; + for (const name of columnNames) { + const values = columns[name]; + const out = new values.constructor(total); + let written = 0; + for (const [start, stop] of spans) { + out.set(values.subarray(start, stop), written); + written += stop - start; + } + rows[name] = out; + } + return rows; + }, + // The sparse virtual image: one value per navigation position, summing + // `valueColumn` over the rows whose rounded (x, y) falls inside the mask. + reduce(mask, xColumn, yColumn, valueColumn) { + const xs = columns[xColumn || 'x']; + const ys = columns[yColumn || 'y']; + const values = valueColumn ? columns[valueColumn] : null; + const width = mask.width, height = mask.height; + const positions = offsets.length - 1; + const out = new Float32Array(positions); + for (let position = 0; position < positions; position++) { + let total = 0; + for (let row = offsets[position]; row < offsets[position + 1]; row++) { + const column = Math.round(xs[row]), line = Math.round(ys[row]); + if (column < 0 || line < 0 || column >= width || line >= height) continue; + if (mask[line * width + column]) total += values ? values[row] : 1; + } + out[position] = total; + } + return out; + }, + }; +} + +// Selection mask for a rectangle, circle or annulus widget dict as it appears +// in `overlay_widgets`, in image pixels. A pixel belongs to the mask when its +// integer coordinate lies inside the shape: `x <= column < x + w` for a +// rectangle, `distance <= r` for a circle, `r_inner <= distance <= r_outer` +// for an annulus. The returned array carries `width` and `height` so a +// consumer can index it without being told the grid again. +function maskFromWidget(widget, width, height) { + const mask = new Uint8Array(width * height); + mask.width = width; + mask.height = height; + const type = widget && widget.type; + if (type === 'rectangle') { + const left = Math.max(0, Math.ceil(widget.x)); + const top = Math.max(0, Math.ceil(widget.y)); + const right = Math.min(width - 1, Math.ceil(widget.x + widget.w) - 1); + const bottom = Math.min(height - 1, Math.ceil(widget.y + widget.h) - 1); + for (let line = top; line <= bottom; line++) + for (let column = left; column <= right; column++) mask[line * width + column] = 1; + return mask; + } + if (type === 'circle' || type === 'annular') { + const centreX = widget.cx, centreY = widget.cy; + const outer = type === 'circle' ? widget.r : widget.r_outer; + const inner = type === 'circle' ? 0 : widget.r_inner; + const outerSquared = outer * outer, innerSquared = inner * inner; + const top = Math.max(0, Math.floor(centreY - outer)); + const bottom = Math.min(height - 1, Math.ceil(centreY + outer)); + const left = Math.max(0, Math.floor(centreX - outer)); + const right = Math.min(width - 1, Math.ceil(centreX + outer)); + for (let line = top; line <= bottom; line++) { + const dy = line - centreY; + for (let column = left; column <= right; column++) { + const dx = column - centreX; + const distanceSquared = dx * dx + dy * dy; + if (distanceSquared <= outerSquared && distanceSquared >= innerSquared) + mask[line * width + column] = 1; + } + } + return mask; + } + throw new Error(`maskFromWidget: unsupported widget type ${type}`); +} + +// Splat `{x, y, intensity}` rows as filled disks into a Float32 image. This +// is how a vectors panel gets a base image: `combine` is "max" for one +// position's rows and "sum" when several positions were gathered. +function rasterDisks(rows, width, height, radius, combine) { + const out = new Float32Array(width * height); + const xs = rows.x, ys = rows.y; + const intensity = rows.intensity || null; + const accumulate = combine === 'sum'; + const reach = Math.ceil(radius); + const radiusSquared = radius * radius; + for (let row = 0; row < xs.length; row++) { + const centreX = Math.round(xs[row]), centreY = Math.round(ys[row]); + const value = intensity ? intensity[row] : 1; + for (let dy = -reach; dy <= reach; dy++) { + const line = centreY + dy; + if (line < 0 || line >= height) continue; + for (let dx = -reach; dx <= reach; dx++) { + if (dx * dx + dy * dy > radiusSquared) continue; + const column = centreX + dx; + if (column < 0 || column >= width) continue; + const at = line * width + column; + out[at] = accumulate ? out[at] + value : Math.max(out[at], value); + } + } + } + return out; +} + +// Percentile display window over the finite values, as [low, high]. `lo` and +// `hi` are percentages; 0 and 100 return the exact extremes, anything between +// is read off a 1024-bin histogram so the cost stays one pass over the data +// rather than a sort. +function robustLevels(values, lo, hi) { + const lowPercent = lo === undefined ? 2 : lo; + const highPercent = hi === undefined ? 98 : hi; + const BINS = 1024; + let minimum = Infinity, maximum = -Infinity, finite = 0; + for (let i = 0; i < values.length; i++) { + const value = values[i]; + if (!Number.isFinite(value)) continue; + if (value < minimum) minimum = value; + if (value > maximum) maximum = value; + finite++; + } + if (!finite) return [0, 1]; + if (maximum <= minimum) return [minimum, minimum + 1]; + const span = maximum - minimum; + const histogram = new Int32Array(BINS); + const scale = BINS / span; + for (let i = 0; i < values.length; i++) { + const value = values[i]; + if (!Number.isFinite(value)) continue; + const bin = Math.min(BINS - 1, Math.floor((value - minimum) * scale)); + histogram[bin]++; + } + function percentile(percent) { + if (percent <= 0) return minimum; + if (percent >= 100) return maximum; + const target = (percent / 100) * finite; + let seen = 0; + for (let bin = 0; bin < BINS; bin++) { + seen += histogram[bin]; + if (seen >= target) return minimum + ((bin + 0.5) * span) / BINS; + } + return maximum; + } + const low = percentile(lowPercent), high = percentile(highPercent); + return high > low ? [low, high] : [minimum, maximum]; +} + +// Map values onto the 8-bit colormap codes the renderer blits. Matches the +// Python quantiser: clip into [0, 255], then truncate. An infinity saturates +// at whichever end it lies past; a NaN fails every comparison and becomes 0. +function toU8(values, vmin, vmax) { + const out = new Uint8Array(values.length); + const scale = 255 / ((vmax - vmin) || 1); + for (let i = 0; i < values.length; i++) { + const code = (values[i] - vmin) * scale; + out[i] = code > 255 ? 255 : (code > 0 ? code : 0); + } + return out; +} + +// ── page chrome ──────────────────────────────────────────────────────────── + +// Forward touches to the renderer's mouse handlers. The draw path listens for +// mousedown/mousemove/mouseup only, so without this a phone can see the page +// but cannot drag a widget. +function installTouchShim(el) { + function forward(touchEvent, mouseType) { + const touch = touchEvent.changedTouches[0]; + if (!touch) return; + const target = document.elementFromPoint(touch.clientX, touch.clientY) || el; + target.dispatchEvent(new MouseEvent(mouseType, { + bubbles: true, cancelable: true, view: window, + clientX: touch.clientX, clientY: touch.clientY, buttons: 1, + })); + touchEvent.preventDefault(); + } + el.addEventListener('touchstart', (e) => forward(e, 'mousedown'), { passive: false }); + el.addEventListener('touchmove', (e) => forward(e, 'mousemove'), { passive: false }); + el.addEventListener('touchend', (e) => forward(e, 'mouseup'), { passive: false }); + el.addEventListener('touchcancel', (e) => forward(e, 'mouseup'), { passive: false }); +} + +// Tell a host iframe how tall the page is, so it can size itself to the +// content instead of guessing or scrolling. +function reportEmbedHeight() { + function send() { + try { + const height = Math.ceil(document.documentElement.scrollHeight); + if (window.parent && window.parent !== window) + window.parent.postMessage({ aplEmbedHeight: height }, '*'); + } catch (_) {} + } + send(); + if (typeof ResizeObserver !== 'undefined') + new ResizeObserver(send).observe(document.documentElement); + window.addEventListener('resize', send); } +// ── dispatch ─────────────────────────────────────────────────────────────── + +// A 3-D panel's cloud rides `panel__geom` as base64: the binary +// side-table path is registered for image pixels only, and a cloud changes on +// a view click rather than per navigator move, so the encode is not on a hot +// path. +function encodeBase64(bytes) { + const CHUNK = 0x8000; + let binary = ''; + for (let at = 0; at < bytes.length; at += CHUNK) + binary += String.fromCharCode.apply(null, bytes.subarray(at, at + CHUNK)); + return btoa(binary); +} +function typedArrayBytes(array) { + return new Uint8Array(array.buffer, array.byteOffset, array.byteLength); +} +// Marker and extra-line groups the dispatch writes carry this prefix, so a +// refresh can tell its own groups from the page's. +const OVERLAY_ID_PREFIX = 'apl-overlay-'; +function indexList(index) { + if (index === null || index === undefined) return []; + if (Array.isArray(index)) return Array.isArray(index[0]) ? index : [index]; + return [[index]]; +} +// A 1-D panel's x axis travels as base64 (`x_axis_b64`), and `draw1d` caches +// the decoded copy on the panel, so read that and fall back to decoding. +function panelAxis(panel) { + if (panel && panel._1dXArr && panel._1dXArr.length) return panel._1dXArr; + const state = (panel && panel.state) || {}; + if (state.x_axis_b64) { + const binary = atob(state.x_axis_b64); + const bytes = new Uint8Array(binary.length); + for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); + return new Float64Array(bytes.buffer); + } + return state.x_axis || []; +} +function nearestAxisIndex(axis, value) { + if (!axis || !axis.length) return 0; + let best = 0, bestDistance = Infinity; + for (let i = 0; i < axis.length; i++) { + const distance = Math.abs(axis[i] - value); + if (distance < bestDistance) { bestDistance = distance; best = i; } + } + return best; +} + +// Mount a navigated page and wire its navigator widgets to its bindings. +// Resolves to the `mount()` handle extended with `dispatch(index)`, the +// current `index`, and the decoded `blocks`. +export async function mountNavigated(el, page, opts) { + const options = opts || {}; + const bindings = page.bindings || []; + const chrome = page.chrome || {}; + const blocks = await decodeBlocks(page.blocks || {}); + + const readers = new Map(); + function readerFor(name) { + if (!readers.has(name)) { + const block = blocks[name]; + if (!block) throw new Error(`unknown block ${name}`); + readers.set(name, block.kind === 'ragged' ? ragged(block) : dense(block)); + } + return readers.get(name); + } + + // Both kinds of refresh coalesce onto one animation frame, latest wins: a + // drag fires an event per pointer move, and each one is a pass over a block. + let queuedIndex = null; + let dispatchRequest = null; + let queuedReduce = null; + let reduceRequest = null; + + const handle = mount(el, page.state, Object.assign({}, options, { + onEvent(event) { + handleEvent(event); + if (options.onEvent) options.onEvent(event); + }, + })); + + function panelFor(panelId) { + return (handle.api.panels && handle.api.panels.get(panelId)) || null; + } + + function panelState(panelId) { + const panel = panelFor(panelId); + return (panel && panel.state) || {}; + } + + function bindingFor(panelId) { + return bindings.find((binding) => binding.panel_id === panelId) || null; + } + + // A navigator widget's position, as a navigation index or a set of them. + // The navigator panel IS the navigation grid, so a 2-D widget's image pixels + // are already the index and a 1-D widget's data coordinate resolves through + // the panel's own x axis. + function indexFromWidget(binding, widget) { + const state = panelState(binding.panel_id); + const columns = state.image_width || 1, lines = state.image_height || 1; + if (widget.type === 'crosshair') + return [Math.round(widget.cy), Math.round(widget.cx)]; + if (widget.type === 'rectangle') { + const maxColumns = widget.max_w == null ? widget.w : Math.min(widget.w, widget.max_w); + const maxLines = widget.max_h == null ? widget.h : Math.min(widget.h, widget.max_h); + const firstColumn = Math.max(0, Math.round(widget.x)); + const firstLine = Math.max(0, Math.round(widget.y)); + const lastColumn = Math.min(columns - 1, Math.round(widget.x + maxColumns) - 1); + const lastLine = Math.min(lines - 1, Math.round(widget.y + maxLines) - 1); + const indices = []; + for (let line = firstLine; line <= lastLine; line++) + for (let column = firstColumn; column <= lastColumn; column++) indices.push([line, column]); + return indices.length ? indices : [[firstLine, firstColumn]]; + } + const axis = panelAxis(panelFor(binding.panel_id)); + if (widget.type === 'vline' || widget.type === 'point') + return [nearestAxisIndex(axis, widget.x)]; + // The span selector is the 1-D analogue of the rectangle: it selects a + // run of positions, and its own max_extent has already capped the drag. + if (widget.type === 'range') { + const first = nearestAxisIndex(axis, Math.min(widget.x0, widget.x1)); + const last = nearestAxisIndex(axis, Math.max(widget.x0, widget.x1)); + const indices = []; + for (let position = first; position <= last; position++) indices.push([position]); + return indices; + } + return null; + } + + // Which of a `views` binding's entries the page's segmented control is on. + // Held by POSITION: two views may read the same block with different + // colours, which is what a direction toggle over one cloud looks like. + const activeViews = new Map(); + + function activeView(binding) { + if (!binding.views || !binding.views.length) return null; + return binding.views[activeViews.get(binding.panel_id) || 0] || binding.views[0]; + } + + function frameBlock(binding) { + const view = activeView(binding); + if (view) return view.block; + if (binding.frame && binding.frame.block) return binding.frame.block; + throw new Error(`binding for panel ${binding.panel_id} names no frame block`); + } + + // The per-point colours that go with the entry currently shown. + function frameColorsBlock(binding) { + const view = activeView(binding); + if (view && view.colors) return view.colors; + return binding.frame && binding.frame.colors; + } + + function frameValues(binding, indices) { + const reader = readerFor(frameBlock(binding)); + const single = indices.length === 1; + if (binding.frame.kind === 'disks') { + const rows = single ? reader.at(indices[0]) : reader.gather(indices); + const state = panelState(binding.panel_id); + const width = binding.frame.width || state.image_width; + const height = binding.frame.height || state.image_height; + const combine = binding.frame.combine || (single ? 'max' : 'sum'); + return { values: rasterDisks(rows, width, height, binding.frame.radius || 3, combine), + width, height }; + } + const values = single ? reader.at(indices[0]) : reader.gather(indices); + const shape = blocks[frameBlock(binding)].shape; + return { values, width: shape[shape.length - 1], height: shape[shape.length - 2] }; + } + + // The panel's own colour window when it has one: a window recomputed per + // frame both costs two passes over the data and makes the contrast jump + // between neighbouring positions, which reads as the data changing. + function frameLevels(binding, values) { + if (binding.frame && binding.frame.levels) return binding.frame.levels; + const state = panelState(binding.panel_id); + if (Number.isFinite(state.display_min) && Number.isFinite(state.display_max) + && state.display_max > state.display_min) + return [state.display_min, state.display_max]; + return robustLevels(values, 2, 98); + } + + // A cloud is the whole dataset, not one position's frame, so it is pushed + // when the binding first shows it and again when a view click swaps it, not + // on every navigator move. + const shownClouds = new Map(); + + function paintPoints3d(binding) { + const name = frameBlock(binding); + const colorsName = frameColorsBlock(binding); + const shown = `${name}|${colorsName || ''}`; + if (shownClouds.get(binding.panel_id) === shown) return; + shownClouds.set(binding.panel_id, shown); + + const points = blocks[name]; + if (!points || points.kind !== 'dense' || points.shape.length !== 2 + || points.shape[1] !== 3) + throw new Error(`points3d block ${name} must be dense (M, 3), got ` + + `${points ? points.shape : 'nothing'}`); + const count = points.shape[0]; + const depth = new Float32Array(count); + for (let point = 0; point < count; point++) depth[point] = points.array[point * 3 + 2]; + + const geomKey = `panel_${binding.panel_id}_geom`; + let geom = {}; + try { geom = JSON.parse(handle.get(geomKey) || '{}'); } catch (_) {} + geom.vertices_b64 = encodeBase64(typedArrayBytes(points.array)); + geom.z_values_b64 = encodeBase64(typedArrayBytes(depth)); + if (colorsName) { + const colors = blocks[colorsName]; + if (!colors || colors.kind !== 'dense' || colors.shape[0] !== count) + throw new Error(`colors block ${colorsName} must be dense with ` + + `${count} rows, got ${colors ? colors.shape : 'nothing'}`); + geom.point_colors_b64 = encodeBase64(typedArrayBytes(colors.array)); + } + handle.applyUpdate(geomKey, JSON.stringify(geom)); + // vertices_count and a bumped revision live on the LIGHT trait; without + // them the renderer draws the old cloud's point count. + const state = panelState(binding.panel_id); + handle.patchPanel(binding.panel_id, { + vertices_count: count, + _geom_rev: (state._geom_rev || 0) + 1, + }); + } + + function paintFrame(binding, indices) { + const panel = panelFor(binding.panel_id); + if ((binding.frame && binding.frame.kind === 'points3d') + || (panel && panel.kind === '3d')) { + paintPoints3d(binding); + return; + } + const { values, width, height } = frameValues(binding, indices); + const levels = frameLevels(binding, values); + handle.setImage(binding.panel_id, toU8(values, levels[0], levels[1]), width, height, + { display_min: levels[0], display_max: levels[1] }); + } + + // One overlay entry → a marker group for a 2-D panel, or an extra line for a + // 1-D one. Columns default to x/y (and u/v, x1/y1/x2/y2, value) so a plain + // block needs no column map. + function overlayWire(overlay, rows) { + const style = overlay.style || {}; + const names = overlay.columns || {}; + const xs = rows[names.x || 'x'], ys = rows[names.y || 'y']; + const count = xs ? xs.length : 0; + // The style IS the wire dict (the keys MarkerGroup.to_wire emits), carried + // through whole: an allow-list here silently drops whatever it has not + // heard of, and `fill_alpha: 0` reads as "unset" to the renderer's default. + const wire = Object.assign({}, style); + delete wire.radius; // an input for `sizes`, not a wire field + wire.id = `apl-overlay-${overlay.block}`; + wire.name = overlay.block; + if (wire.color === undefined) wire.color = '#ff0000'; + if (overlay.kind === 'curves') { + wire.data = Array.from(rows[names.value || 'value'] || []); + wire.x_axis = Array.from(xs || []); + if (wire.linewidth === undefined) wire.linewidth = 1.5; + if (wire.linestyle === undefined) wire.linestyle = 'solid'; + if (wire.alpha === undefined) wire.alpha = 1; + if (wire.marker === undefined) wire.marker = 'none'; + if (wire.markersize === undefined) wire.markersize = 4; + if (wire.label === undefined) wire.label = ''; + if (wire.axis === undefined) wire.axis = 'left'; + return wire; + } + const offsets = []; + for (let row = 0; row < count; row++) offsets.push([xs[row], ys[row]]); + wire.type = overlay.kind; + if (wire.linewidth === undefined) wire.linewidth = 1.5; + if (overlay.kind === 'circles') { + wire.offsets = offsets; + const radiusColumn = names.radius ? rows[names.radius] : null; + const radius = style.radius === undefined ? 5 : style.radius; + wire.sizes = offsets.map((_, row) => (radiusColumn ? radiusColumn[row] : radius)); + } else if (overlay.kind === 'arrows') { + wire.offsets = offsets; + const us = rows[names.u || 'u'], vs = rows[names.v || 'v']; + wire.U = offsets.map((_, row) => (us ? us[row] : 0)); + wire.V = offsets.map((_, row) => (vs ? vs[row] : 0)); + } else if (overlay.kind === 'lines') { + const x1 = rows[names.x1 || 'x1'], y1 = rows[names.y1 || 'y1']; + const x2 = rows[names.x2 || 'x2'], y2 = rows[names.y2 || 'y2']; + const segments = []; + for (let row = 0; row < (x1 ? x1.length : 0); row++) + segments.push([[x1[row], y1[row]], [x2[row], y2[row]]]); + wire.segments = segments; + } else { + throw new Error(`unsupported overlay kind ${overlay.kind}`); + } + return wire; + } + + // Keep every group the figure was built with, replacing only the ones this + // runtime owns: a dispatch that assigned the list wholesale would wipe the + // page's own annotations on the first crosshair move. + function mergeOverlayGroups(existing, fresh) { + const kept = (existing || []).filter( + (group) => !String(group && group.id).startsWith(OVERLAY_ID_PREFIX)); + return kept.concat(fresh); + } + + // One 3-D point marked on the panel, optionally turning the camera to face + // it. The rule is not "some angle pointing that way": atan2(y, x) - 90 names + // the same direction 180 degrees out and lands the point on the far edge. + function paintHighlight(binding, overlay, indices) { + const reader = readerFor(overlay.block); + const rows = reader.at(indices[0]); + let x, y, z; + if (reader.kind === 'ragged') { + const names = overlay.columns || {}; + const xs = rows[names.x || 'x'], ys = rows[names.y || 'y']; + const zs = rows[names.z || 'z']; + if (!xs || !xs.length) return; + x = xs[0]; y = ys[0]; z = zs[0]; + } else { + if (rows.length < 3) return; + x = rows[0]; y = rows[1]; z = rows[2]; + } + if (!Number.isFinite(x) || !Number.isFinite(y) || !Number.isFinite(z)) return; + + const patch = { highlight: Object.assign( + { color: '#ff1744', size: 7 }, overlay.style || {}, { x, y, z }) }; + const faceCamera = overlay.face_camera || binding.face_camera; + if (faceCamera) { + const radius = Math.hypot(x, y, z) || 1; + patch.azimuth = Math.atan2(x, -y) * 180 / Math.PI; + patch.elevation = Math.asin(Math.max(-1, Math.min(1, z / radius))) * 180 / Math.PI; + } + // Written either way: the flag persists on the trait, so leaving a + // previous face_camera push's `true` in place would let a later highlight + // discard the orbit the reader is holding. + patch._view_from_python = !!faceCamera; + handle.patchPanel(binding.panel_id, patch); + } + + function paintOverlays(binding, indices) { + const markers = []; + const lines = []; + let anyMarkers = false; + for (const overlay of binding.overlays) { + if (overlay.kind === 'highlight') { paintHighlight(binding, overlay, indices); continue; } + const reader = readerFor(overlay.block); + const rows = indices.length === 1 ? reader.at(indices[0]) : reader.gather(indices); + if (overlay.kind === 'curves') { lines.push(overlayWire(overlay, rows)); continue; } + anyMarkers = true; + markers.push(overlayWire(overlay, rows)); + } + const state = panelState(binding.panel_id); + const patch = {}; + if (anyMarkers) patch.markers = mergeOverlayGroups(state.markers, markers); + if (lines.length) patch.extra_lines = mergeOverlayGroups(state.extra_lines, lines); + if (Object.keys(patch).length) handle.patchPanel(binding.panel_id, patch); + } + + function formatReadout(binding, indices) { + const spec = binding.readout; + const reader = readerFor(spec.block); + const rows = reader.kind === 'dense' + ? { value: reader.at(indices[0]) } : reader.at(indices[0]); + const names = spec.names || Object.keys(rows); + const units = spec.units || {}; + return names.map((name) => { + const values = rows[name]; + const value = values && values.length ? values[0] : NaN; + const unit = units[name] ? ` ${units[name]}` : ''; + return `${name} ${Number.isFinite(value) ? value.toPrecision(4) : '-'}${unit}`; + }).join(' '); + } + + function writeText(id, text) { + const node = document.getElementById(id); + if (node) node.textContent = text; + } + + // Wire the page's segmented control for a `views` binding: each button names + // a block, and picking one re-reads the panel's frame from it at the + // position the navigator is already on. + function installViews(binding) { + activeViews.set(binding.panel_id, 0); + const group = document.getElementById(`apl-views-${binding.panel_id}`); + if (!group) return; + const buttons = [...group.querySelectorAll('button[data-view]')]; + const mark = () => { + const chosen = activeViews.get(binding.panel_id); + for (const button of buttons) + button.setAttribute('aria-pressed', + Number(button.dataset.view) === chosen ? 'true' : 'false'); + }; + for (const button of buttons) + button.addEventListener('click', () => { + activeViews.set(binding.panel_id, Number(button.dataset.view)); + mark(); + if (handle.index !== null) dispatch(handle.index); + }); + mark(); + } + + // Refresh every driven binding for one navigation index (or index set). + function dispatch(index) { + const indices = indexList(index); + if (!indices.length) return; + handle.index = index; + for (const binding of bindings) { + if (binding.role !== 'driven') continue; + if (binding.frame || binding.views) paintFrame(binding, indices); + if (binding.overlays && binding.overlays.length) paintOverlays(binding, indices); + if (binding.readout) + writeText(`apl-readout-${binding.panel_id}`, formatReadout(binding, indices)); + } + } + + function requestDispatch(index) { + queuedIndex = index; + if (dispatchRequest !== null) return; + dispatchRequest = requestAnimationFrame(() => { + dispatchRequest = null; + const next = queuedIndex; + queuedIndex = null; + dispatch(next); + }); + } + + // A detector widget on a driven panel reduces the whole block back onto the + // navigator: the navigator image becomes sum(frame * mask) per position. + // Only these three define a region of the signal grid to sum over; any other + // widget on the same panel is left to do whatever else it is there for. + const DETECTOR_TYPES = ['rectangle', 'circle', 'annular']; + + function paintReduced(binding, widget) { + const spec = binding.reduce; + const state = panelState(binding.panel_id); + const mask = maskFromWidget(widget, state.image_width, state.image_height); + const reader = readerFor(spec.block); + const values = reader.kind === 'ragged' + ? reader.reduce(mask, spec.x, spec.y, spec.value) : reader.reduce(mask); + const navigatorState = panelState(spec.navigator_panel); + const levels = robustLevels(values, 0, 100); + handle.setImage(spec.navigator_panel, toU8(values, levels[0], levels[1]), + navigatorState.image_width, navigatorState.image_height, + { display_min: levels[0], display_max: levels[1] }); + } + + function requestReduce(binding, widget) { + queuedReduce = [binding, widget]; + if (reduceRequest !== null) return; + reduceRequest = requestAnimationFrame(() => { + reduceRequest = null; + const next = queuedReduce; + queuedReduce = null; + paintReduced(next[0], next[1]); + }); + } + + function handleEvent(event) { + if (!event || !event.widget_id) return; + if (event.event_type !== 'pointer_move' && event.event_type !== 'pointer_up') return; + const binding = bindingFor(event.panel_id); + if (!binding) return; + if (binding.role === 'navigator') { + const index = indexFromWidget(binding, event); + if (index) requestDispatch(index); + return; + } + if (binding.role === 'driven' && binding.reduce + && DETECTOR_TYPES.includes(event.type)) requestReduce(binding, event); + } + + if (chrome.touch !== false) installTouchShim(el); + if (chrome.height_report !== false) reportEmbedHeight(); + + // A binding may carry its panel's widgets, so a page can declare the + // navigator's crosshair or detector next to what it drives. + for (const binding of bindings) + if (binding.widgets) handle.patchPanel(binding.panel_id, + { overlay_widgets: binding.widgets }); + + for (const binding of bindings) { + if (binding.views && binding.views.length) installViews(binding); + if (!binding.reduce) continue; + // A reduce binding with no detector on its panel can never fire, and the + // page author has no other way to find that out. + const widgets = panelState(binding.panel_id).overlay_widgets || []; + if (!widgets.some((widget) => DETECTOR_TYPES.includes(widget.type))) + throw new Error(`panel ${binding.panel_id} reduces onto ` + + `${binding.reduce.navigator_panel} but carries no ` + + `${DETECTOR_TYPES.join('/')} widget to reduce under`); + } + + handle.dispatch = dispatch; + handle.blocks = blocks; + handle.index = null; + + const navigatorBinding = bindings.find((binding) => binding.role === 'navigator'); + const navigatorState = navigatorBinding ? panelState(navigatorBinding.panel_id) : {}; + const initial = (navigatorBinding && navigatorBinding.initial_index) + || (navigatorState.image_height ? [0, 0] : [0]); + dispatch(initial); + return handle; +} + +// The readers are one namespace rather than seven top-level exports: they are +// generic names (`dense`, `ragged`, `toU8`) that would otherwise sit beside +// `mount` and `render` in every importer's completion list. +export const embed = { + mountNavigated, decodeBlocks, dense, ragged, + maskFromWidget, rasterDisks, robustLevels, toU8, +}; diff --git a/anyplotlib/tests/test_embed/_export_utils.py b/anyplotlib/tests/test_embed/_export_utils.py index f2be65e0..a4d384e3 100644 --- a/anyplotlib/tests/test_embed/_export_utils.py +++ b/anyplotlib/tests/test_embed/_export_utils.py @@ -22,8 +22,11 @@ const STATE = __STATE__; const esmSource = __ESM__; const blobUrl = URL.createObjectURL(new Blob([esmSource], {type: "text/javascript"})); +window._syncs = []; import(blobUrl).then(mod => { - window._handle = mod.mount(document.getElementById("host"), STATE, {}); + window._handle = mod.mount(document.getElementById("host"), STATE, { + onSync: (key, value) => window._syncs.push({key, value}), + }); window._aplReady = true; }).catch(err => { document.body.textContent = "mount error: " + err; }); diff --git a/anyplotlib/tests/test_embed/test_embed_3d.py b/anyplotlib/tests/test_embed/test_embed_3d.py new file mode 100644 index 00000000..f9f4dc32 --- /dev/null +++ b/anyplotlib/tests/test_embed/test_embed_3d.py @@ -0,0 +1,328 @@ +""" +A 3-D panel driven by a navigated page. + +The shape this exists for is an orientation map: a navigator of orientations, +a sphere of the whole cloud beside it, and a highlight marking the position +under the crosshair. The sphere turns to face the picked point, and a +direction toggle swaps the cloud and its per-point colours. +""" +from __future__ import annotations + +import math +import pathlib +import tempfile + +import numpy as np +import pytest + +import anyplotlib as apl +from anyplotlib.embed import navigated_html + +NAV_SHAPE = (6, 6) + + +def _sphere_dataset(): + """A unit-sphere cloud, two colourings of it, and one direction per position.""" + rng = np.random.default_rng(17) + count = 240 + points = rng.normal(size=(count, 3)).astype(np.float32) + points /= np.linalg.norm(points, axis=1, keepdims=True) + + red = np.zeros((count, 3), dtype=np.uint8) + red[:, 0] = 230 + blue = np.zeros((count, 3), dtype=np.uint8) + blue[:, 2] = 230 + + positions = NAV_SHAPE[0] * NAV_SHAPE[1] + picked = points[rng.integers(0, count, size=positions)].astype(np.float32) + return points, red, blue, picked.reshape(NAV_SHAPE + (3,)) + + +def _sphere_page(picks, *, face_camera=True, with_views=True): + points, red, blue, _ = _sphere_dataset() + fig, axes = apl.subplots(1, 2, figsize=(640, 320)) + navigator = axes[0].imshow(picks[..., 2].astype(np.float32), cmap="gray") + sphere = axes[1].scatter3d(points[:, 0], points[:, 1], points[:, 2], + colors=np.zeros((len(points), 3), dtype=np.float32), + point_size=4.0, azimuth=-60.0, elevation=30.0) + sphere.set_sphere(1.0) + navigator.add_widget("crosshair", cx=0, cy=0) + + driven = {"panel_id": sphere._id, "role": "driven", + "overlays": [{"block": "picks", "kind": "highlight", + "style": {"color": "#ffffff", "size": 11}, + "face_camera": face_camera}]} + if with_views: + driven["views"] = [{"label": "z", "block": "cloud", "colors": "red"}, + {"label": "x", "block": "cloud", "colors": "blue"}] + driven["frame"] = {"kind": "points3d"} + else: + driven["frame"] = {"block": "cloud", "kind": "points3d", "colors": "red"} + + blocks = {"cloud": points, "red": red, "blue": blue, "picks": picks} + html = navigated_html( + fig, blocks, + [{"panel_id": navigator._id, "role": "navigator"}, driven]) + return html, navigator._id, sphere._id + + +@pytest.fixture +def sphere_page(_pw_browser): + """Open a navigated page holding a 3-D panel; return the live Page.""" + pages, paths = [], [] + + def _open(html): + with tempfile.NamedTemporaryFile( + suffix=".html", mode="w", encoding="utf-8", delete=False) as handle: + handle.write(html) + path = pathlib.Path(handle.name) + paths.append(path) + page = _pw_browser.new_page() + pages.append(page) + page.goto(path.as_uri()) + page.wait_for_function("() => window._aplReady === true", timeout=20_000) + page.evaluate( + "() => new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))") + return page + + yield _open + for page in pages: + try: + page.close() + except Exception: + pass + for path in paths: + path.unlink(missing_ok=True) + + +_CLIENT_POINT = """ +(args) => { + const [panelId, ix, iy] = args; + const panel = window._aplHandle.api.panels.get(panelId); + const state = panel.state; + const scale = Math.min(panel.imgW / state.image_width, + panel.imgH / state.image_height); + const fitWidth = state.image_width * scale, fitHeight = state.image_height * scale; + const x = (panel.imgW - fitWidth) / 2 + (ix + 0.5) / state.image_width * fitWidth; + const y = (panel.imgH - fitHeight) / 2 + (iy + 0.5) / state.image_height * fitHeight; + const rect = panel.overlayCanvas.getBoundingClientRect(); + return [rect.left + x * (rect.width / panel.imgW), + rect.top + y * (rect.height / panel.imgH)]; +} +""" + + +def _drag(page, panel_id, start, end): + x0, y0 = page.evaluate(_CLIENT_POINT, [panel_id, *start]) + x1, y1 = page.evaluate(_CLIENT_POINT, [panel_id, *end]) + page.mouse.move(x0, y0) + page.mouse.down() + page.mouse.move(x1, y1, steps=4) + page.mouse.up() + page.evaluate( + "() => new Promise(r => requestAnimationFrame(" + " () => requestAnimationFrame(() => requestAnimationFrame(r))))") + + +def _camera(page, panel_id): + return page.evaluate( + "(id) => { const s = window._aplHandle.api.panels.get(id).state;" + " return [s.azimuth, s.elevation, s._view_from_python]; }", + panel_id) + + +def _ink(page, panel_id, channel): + """Pixels on the 3-D plot canvas whose dominant channel is *channel*.""" + return page.evaluate( + """(args) => { + const [id, channel] = args; + const canvas = window._aplHandle.api.panels.get(id).plotCanvas; + const data = canvas.getContext('2d') + .getImageData(0, 0, canvas.width, canvas.height).data; + let n = 0; + for (let i = 0; i < data.length; i += 4) { + if (data[i + 3] < 200) continue; + const rgb = [data[i], data[i + 1], data[i + 2]]; + const best = rgb.indexOf(Math.max(...rgb)); + if (best === channel && rgb[best] - Math.min(...rgb) > 60) n++; + } + return n; + }""", + [panel_id, channel]) + + +class TestHighlightFollowsTheNavigator: + def test_the_highlight_is_the_block_row_at_that_index(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + + highlight = page.evaluate( + "(id) => window._aplHandle.api.panels.get(id).state.highlight", sphere_id) + assert highlight["color"] == "#ffffff" and highlight["size"] == 11 + assert np.allclose([highlight["x"], highlight["y"], highlight["z"]], + picks[0, 0], atol=1e-6) + + _drag(page, navigator_id, (0, 0), (4, 2)) + assert page.evaluate("() => window._aplHandle.index") == [2, 4] + moved = page.evaluate( + "(id) => window._aplHandle.api.panels.get(id).state.highlight", sphere_id) + assert np.allclose([moved["x"], moved["y"], moved["z"]], + picks[2, 4], atol=1e-6) + + def test_the_marked_point_repaints(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + + def white_pixels(): + return page.evaluate( + """(id) => { + const canvas = window._aplHandle.api.panels.get(id).plotCanvas; + const data = canvas.getContext('2d') + .getImageData(0, 0, canvas.width, canvas.height).data; + const out = []; + for (let i = 0; i < data.length; i += 4) + if (data[i] > 240 && data[i + 1] > 240 && data[i + 2] > 240 + && data[i + 3] > 250) out.push(i >> 2); + return out; + }""", + sphere_id) + + before = white_pixels() + assert before, "the highlight drew nothing" + _drag(page, navigator_id, (0, 0), (5, 5)) + after = white_pixels() + assert after, "the highlight vanished" + assert before != after, "the highlight did not move with the navigator" + + +class TestFaceCamera: + def test_the_camera_turns_to_the_picked_point(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=True) + page = sphere_page(html) + _drag(page, navigator_id, (0, 0), (3, 1)) + + x, y, z = (float(value) for value in picks[1, 3]) + radius = math.hypot(x, y, z) or 1.0 + azimuth, elevation, from_python = _camera(page, sphere_id) + assert from_python is True + assert azimuth == pytest.approx(math.degrees(math.atan2(x, -y)), abs=1e-6) + assert elevation == pytest.approx(math.degrees(math.asin(z / radius)), abs=1e-6) + + def test_without_it_the_readers_orbit_is_kept(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + + # Orbit the sphere the way a reader would, then move the navigator. + page.evaluate( + """async (id) => { + const panel = window._aplHandle.api.panels.get(id); + const rect = panel.overlayCanvas.getBoundingClientRect(); + const mid = {clientX: rect.left + rect.width / 2, + clientY: rect.top + rect.height / 2}; + panel.overlayCanvas.dispatchEvent( + new MouseEvent('mousedown', {bubbles: true, ...mid})); + document.dispatchEvent(new MouseEvent('mousemove', { + bubbles: true, clientX: mid.clientX + 70, clientY: mid.clientY + 25})); + document.dispatchEvent(new MouseEvent('mouseup', {bubbles: true})); + await new Promise((r) => requestAnimationFrame(r)); + }""", + sphere_id) + orbited = _camera(page, sphere_id) + assert orbited[:2] != [-60.0, 30.0], "the orbit did not move the camera" + + _drag(page, navigator_id, (0, 0), (2, 3)) + after = _camera(page, sphere_id) + assert after[0] == pytest.approx(orbited[0]), after + assert after[1] == pytest.approx(orbited[1]), after + assert after[2] is False + + +class TestViewsSwapTheCloud: + def test_a_view_click_repaints_the_points_in_its_colours(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + + red_before, blue_before = _ink(page, sphere_id, 0), _ink(page, sphere_id, 2) + assert red_before > 50, f"the cloud did not paint red: {red_before}" + + page.evaluate( + """async (id) => { + document.querySelector(`#apl-views-${id} button[data-view="1"]`) + .click(); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + sphere_id) + red_after, blue_after = _ink(page, sphere_id, 0), _ink(page, sphere_id, 2) + assert blue_after > blue_before, (red_before, blue_before, red_after, blue_after) + assert red_after < red_before, (red_before, blue_before, red_after, blue_after) + + def test_the_cloud_is_pushed_once_not_per_move(self, sphere_page): + """A cloud is the whole dataset; only a view click changes it.""" + _, _, _, picks = _sphere_dataset() + html, navigator_id, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + first = page.evaluate( + "(id) => JSON.parse(window._aplHandle.get(`panel_${id}_json`))._geom_rev", + sphere_id) + _drag(page, navigator_id, (0, 0), (4, 4)) + again = page.evaluate( + "(id) => JSON.parse(window._aplHandle.get(`panel_${id}_json`))._geom_rev", + sphere_id) + assert again == first, "the cloud was re-pushed on a navigator move" + + def test_the_cloud_reaches_the_geometry_channel(self, sphere_page): + points, red, _, picks = _sphere_dataset() + html, _, sphere_id = _sphere_page(picks, with_views=False, face_camera=False) + page = sphere_page(html) + pushed = page.evaluate( + """(id) => { + const geom = JSON.parse(window._aplHandle.get(`panel_${id}_geom`)); + const decode = (text) => { + const binary = atob(text); + const bytes = new Uint8Array(binary.length); + for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i); + return bytes; + }; + const state = window._aplHandle.api.panels.get(id).state; + return {vertices: Array.from(new Float32Array( + decode(geom.vertices_b64).buffer).slice(0, 6)), + colors: Array.from(decode(geom.point_colors_b64).slice(0, 6)), + count: state.vertices_count}; + }""", + sphere_id) + assert pushed["count"] == len(points) + assert np.allclose(pushed["vertices"], points[:2].ravel(), atol=1e-6) + assert pushed["colors"] == list(red[:2].ravel()) + + +class TestSetImageStillRefuses3d: + def test_a_sphere_panel_refuses_pixel_bytes(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, _, sphere_id = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + message = page.evaluate( + """(id) => { + try { + window._aplHandle.setImage(id, new Uint8Array(64), 8, 8); + return null; + } catch (e) { return String(e.message); } + }""", + sphere_id) + assert message is not None and "only a 2-D image panel" in message + + +class TestDenseVectorRead: + def test_at_returns_the_three_vector(self, sphere_page): + _, _, _, picks = _sphere_dataset() + html, _, _ = _sphere_page(picks, face_camera=False) + page = sphere_page(html) + vector = page.evaluate( + "() => Array.from(window._aplModule.embed.dense(" + " window._aplHandle.blocks.picks).at([3, 2]))") + assert np.allclose(vector, picks[3, 2], atol=1e-6) diff --git a/anyplotlib/tests/test_embed/test_embed_api.py b/anyplotlib/tests/test_embed/test_embed_api.py index 5dbb421e..a73ed25d 100644 --- a/anyplotlib/tests/test_embed/test_embed_api.py +++ b/anyplotlib/tests/test_embed/test_embed_api.py @@ -1,20 +1,23 @@ """ Unit tests for anyplotlib.embed — kernel-free embedding API. -Covers figure_state / to_html / save_html / esm_path / Figure.to_html and -the transport-agnostic FigureBridge (outbound forwarding, inbound event -dispatch, echo suppression, dynamic panel traits). +Covers figure_state / to_html / save_html / esm_path / Figure.to_html, the +transport-agnostic FigureBridge (outbound forwarding, inbound event dispatch, +echo suppression, dynamic panel traits), and the navigated-page builders +pack_blocks / navigated_html. """ from __future__ import annotations import json +import re import numpy as np import pytest import anyplotlib as apl from anyplotlib.embed import ( - FigureBridge, esm_path, figure_state, save_html, to_html, + FigureBridge, Ragged, esm_path, figure_state, navigated_html, pack_blocks, + save_html, to_html, ) @@ -131,3 +134,136 @@ def test_close_stops_forwarding(self): bridge.close() plot.set_title("after close") assert sent == [] + + +class TestPackBlocks: + def test_dense_round_trips_bit_exactly(self): + rng = np.random.default_rng(3) + first = rng.integers(0, 256, size=(4, 5, 6)).astype(np.uint8) + second = rng.random((3, 7)).astype(np.float32) + payload, manifest = pack_blocks({"first": first, "second": second}) + + assert manifest["first"]["kind"] == "dense" + assert manifest["first"]["shape"] == [4, 5, 6] + assert manifest["second"]["dtype"] == "float32" + for name, array in (("first", first), ("second", second)): + entry = manifest[name] + raw = payload[entry["offset"]:entry["offset"] + entry["nbytes"]] + restored = np.frombuffer(raw, dtype=array.dtype).reshape(array.shape) + assert np.array_equal(restored, array) + + def test_every_block_starts_on_an_aligned_offset(self): + """A typed-array view can only start on a multiple of its item size.""" + payload, manifest = pack_blocks({ + "odd": np.zeros(5, dtype=np.uint8), + "wide": np.arange(4, dtype=np.float64), + }) + assert manifest["wide"]["offset"] % 8 == 0 + assert len(payload) >= manifest["wide"]["offset"] + manifest["wide"]["nbytes"] + + def test_ragged_round_trips_bit_exactly(self): + offsets = np.array([0, 2, 2, 5], dtype=np.int32) + columns = {"x": np.array([1.5, 2.5, 3.5, 4.5, 5.5], dtype=np.float32), + "intensity": np.arange(5, dtype=np.float32)} + payload, manifest = pack_blocks({ + "spots": Ragged(offsets=offsets, columns=columns, nav_shape=(1, 3))}) + + entry = manifest["spots"] + assert entry["kind"] == "ragged" and entry["nav_shape"] == [1, 3] + raw = payload[entry["offsets"]["offset"]: + entry["offsets"]["offset"] + entry["offsets"]["nbytes"]] + assert np.array_equal(np.frombuffer(raw, dtype=np.int32), offsets) + for name, values in columns.items(): + spec = entry["columns"][name] + raw = payload[spec["offset"]:spec["offset"] + spec["nbytes"]] + assert np.array_equal(np.frombuffer(raw, dtype=np.float32), values) + + def test_unviewable_dtype_is_refused(self): + with pytest.raises(ValueError, match="cannot be viewed"): + pack_blocks({"big": np.zeros(3, dtype=np.int64)}) + + +class TestNavigatedHtml: + def _figure(self): + fig, axes = apl.subplots(1, 2, figsize=(400, 200)) + navigator = axes[0].imshow(np.zeros((4, 4), dtype=np.uint8)) + signal = axes[1].imshow(np.zeros((8, 8), dtype=np.uint8)) + return fig, navigator, signal + + def _bindings(self, navigator, signal): + return [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal._id, "role": "driven", + "frame": {"block": "cube", "kind": "image"}}] + + def test_page_is_self_contained(self): + fig, navigator, signal = self._figure() + cube = np.zeros((4, 4, 8, 8), dtype=np.uint8) + html = navigated_html(fig, {"cube": cube}, + self._bindings(navigator, signal), + title="Scan", caption="drag the crosshair") + assert html.startswith("") + assert "mountNavigated" in html + assert "export async function mountNavigated" in html # the renderer, inlined + assert re.search(r"https?://", html.replace("http://www.w3.org", "")) is None + assert "Scan" in html and "drag the crosshair" in html + + def test_unknown_panel_is_refused(self): + fig, navigator, signal = self._figure() + with pytest.raises(ValueError, match="unknown panel"): + navigated_html(fig, {"cube": np.zeros((4, 4, 8, 8), dtype=np.uint8)}, + [{"panel_id": "nope", "role": "navigator"}]) + + def test_unknown_block_is_refused(self): + fig, navigator, signal = self._figure() + bindings = self._bindings(navigator, signal) + bindings[1]["frame"]["block"] = "missing" + with pytest.raises(ValueError, match="unknown block"): + navigated_html(fig, {"cube": np.zeros((4, 4, 8, 8), dtype=np.uint8)}, + bindings) + + def test_state_dict_is_accepted_in_place_of_a_figure(self): + fig, navigator, signal = self._figure() + html = navigated_html(figure_state(fig), + {"cube": np.zeros((4, 4, 8, 8), dtype=np.uint8)}, + self._bindings(navigator, signal)) + assert f"panel_{signal._id}_json" in html + + +class TestEmbedSurface: + def test_the_readers_live_under_one_namespace(self): + """Generic names stay out of the top level beside mount / render.""" + source = esm_path().read_text(encoding="utf-8") + assert "export const embed = {" in source + assert "export async function mountNavigated" in source + for name in ("dense", "ragged", "toU8", "decodeBlocks", + "maskFromWidget", "rasterDisks", "robustLevels"): + assert f"export function {name}(" not in source, ( + f"{name} is still a top-level export") + assert f"export async function {name}(" not in source + + def test_views_blocks_are_validated(self): + fig, axes = apl.subplots(1, 2, figsize=(400, 200)) + navigator = axes[0].imshow(np.zeros((4, 4), dtype=np.uint8)) + panel = axes[1].imshow(np.zeros((8, 8), dtype=np.uint8)) + cube = np.zeros((4, 4, 8, 8), dtype=np.uint8) + bindings = [ + {"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "views": [{"label": "a", "block": "cube"}, + {"label": "b", "block": "gone"}]}, + ] + with pytest.raises(ValueError, match="unknown block 'gone'"): + navigated_html(fig, {"cube": cube}, bindings) + + def test_a_views_binding_needs_no_frame_block(self): + fig, axes = apl.subplots(1, 2, figsize=(400, 200)) + navigator = axes[0].imshow(np.zeros((4, 4), dtype=np.uint8)) + panel = axes[1].imshow(np.zeros((8, 8), dtype=np.uint8)) + cube = np.zeros((4, 4, 8, 8), dtype=np.uint8) + html = navigated_html( + fig, {"cube": cube}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "views": [{"label": "a", "block": "cube"}], + "frame": {"kind": "image"}}]) + assert f'id="apl-views-{panel._id}"' in html diff --git a/anyplotlib/tests/test_embed/test_embed_escaping.py b/anyplotlib/tests/test_embed/test_embed_escaping.py new file mode 100644 index 00000000..aebaa9ee --- /dev/null +++ b/anyplotlib/tests/test_embed/test_embed_escaping.py @@ -0,0 +1,139 @@ +""" +A self-contained page inlines its data into a ``" + + +class TestScriptJson: + def test_no_script_ender_survives(self): + text = script_json({"title": PAYLOAD, "note": ""}) + assert "", "n": [1, 2.5, None]} + assert json.loads(script_json(value)) == value + + def test_ordinary_values_are_untouched(self): + assert script_json({"a": 1}) == json.dumps({"a": 1}) + + +def _injecting_page(): + """A figure whose title, labels and one binding style carry the payload.""" + rng = np.random.default_rng(1) + block = rng.integers(0, 256, size=(4, 4, 8, 8)).astype(np.uint8) + spots = np.stack([np.full(4 * 4, 3.0, dtype=np.float32), + np.full(4 * 4, 4.0, dtype=np.float32)]) + + fig, axes = apl.subplots(1, 2, figsize=(520, 260)) + navigator = axes[0].imshow(block.sum(axis=(2, 3)).astype(np.float32), cmap="gray") + panel = axes[1].imshow(block[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + panel.set_title(PAYLOAD) + panel.set_xlabel(PAYLOAD) + + from anyplotlib.embed import Ragged + ragged = Ragged(offsets=np.arange(0, 17, dtype=np.int32), + columns={"x": spots[0], "y": spots[1]}, + nav_shape=(4, 4)) + return navigated_html( + fig, {"cube": block, "spots": ragged}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "frame": {"block": "cube", "kind": "image", "levels": [0, 255]}, + "overlays": [{"block": "spots", "kind": "circles", + "style": {"radius": 3, "color": "#0f0", "label": PAYLOAD}}]}]) + + +class TestNavigatedPageIsNotInjectable: + def test_a_payload_in_the_figure_does_not_run(self, _pw_browser, tmp_path): + html = _injecting_page() + path = tmp_path / "injected.html" + path.write_text(html, encoding="utf-8") + + page = _pw_browser.new_page() + try: + page.goto(path.as_uri()) + page.wait_for_function("() => window._aplReady === true", timeout=20_000) + injected = page.evaluate("() => window.__injected") + canvases = page.evaluate( + "() => document.querySelectorAll('#apl-host canvas').length") + finally: + page.close() + assert injected is None, "the payload executed" + assert canvases >= 3, "the figure did not mount" + + def test_a_payload_in_the_title_and_caption_is_shown_as_text( + self, _pw_browser, tmp_path): + rng = np.random.default_rng(2) + block = rng.integers(0, 256, size=(4, 4, 8, 8)).astype(np.uint8) + fig, axes = apl.subplots(1, 2, figsize=(520, 260)) + navigator = axes[0].imshow(block.sum(axis=(2, 3)).astype(np.float32)) + panel = axes[1].imshow(block[0, 0]) + navigator.add_widget("crosshair", cx=0, cy=0) + panel.set_title(PAYLOAD) # the script-block route + html = navigated_html( + fig, {"cube": block}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "frame": {"block": "cube", "kind": "image"}}], + title=PAYLOAD, caption=PAYLOAD) + path = tmp_path / "titled.html" + path.write_text(html, encoding="utf-8") + + page = _pw_browser.new_page() + try: + page.goto(path.as_uri()) + page.wait_for_function("() => window._aplReady === true", timeout=20_000) + injected = page.evaluate("() => window.__injected") + shown = page.evaluate( + "() => [document.querySelector('.apl-title').textContent," + " document.querySelector('.apl-caption').textContent]") + extra = page.evaluate( + "() => document.querySelectorAll('script').length") + finally: + page.close() + assert injected is None, "the payload executed" + assert shown == [PAYLOAD, PAYLOAD], shown + assert extra == 1, f"{extra} script elements; the page has one" + + +class TestStandalonePageIsNotInjectable: + def test_a_panel_title_does_not_close_the_script(self, _pw_browser): + fig, ax = apl.subplots(1, 1, figsize=(320, 240)) + plot = ax.imshow(np.zeros((8, 8), dtype=np.uint8)) + plot.set_title(PAYLOAD) + html = fig.to_html() + assert " +""" + + +class TestPageChrome: + def test_page_reports_its_height_to_a_host_frame(self, _pw_browser, tmp_path): + signal, spots = _dataset() + html, _, _ = _build_page(signal, spots, detector=False) + embed = tmp_path / "embed.html" + embed.write_text(html, encoding="utf-8") + host = tmp_path / "host.html" + host.write_text(_HOST_PAGE.replace("__SRC__", embed.name), encoding="utf-8") + + page = _pw_browser.new_page() + try: + page.goto(host.as_uri()) + page.wait_for_function("() => window._heights.length > 0", timeout=20_000) + heights = page.evaluate("() => window._heights") + finally: + page.close() + assert heights and all(height > 0 for height in heights), heights + + def test_png_harvest_answers(self, navigated_page): + signal, spots = _dataset() + html, _, _ = _build_page(signal, spots, detector=False) + page = navigated_page(html) + result = page.evaluate( + """() => new Promise((resolve) => { + window.addEventListener('message', (e) => { + if (e.data && e.data.type === 'anyplotlib_export_png_result') + resolve({hasUrl: typeof e.data.dataUrl === 'string', + error: e.data.error || null}); + }); + window.postMessage( + {type: 'anyplotlib_export_png', requestId: 'r1', opts: {}}, '*'); + })""") + assert result["error"] is None and result["hasUrl"], result + + +class TestDatalessPage: + def test_a_page_with_no_blocks_still_mounts(self, navigated_page): + fig, ax = apl.subplots(1, 1, figsize=(320, 240)) + plot = ax.imshow(np.zeros((16, 16), dtype=np.uint8), cmap="gray") + html = navigated_html(fig, {}, [{"panel_id": plot._id, "role": "static"}]) + page = navigated_page(html) + assert page.evaluate("() => window._aplHandle.panelIds()") == [plot._id] + + +# ── a 1-D navigator: a stack of frames scrubbed by a line plot ───────────── + +# figure_esm.js's shared plot-area padding, which a 1-D drag has to land inside. +PAD_LEFT, PAD_RIGHT, PAD_TOP, PAD_BOTTOM = 58, 12, 12, 42 + +_ONE_D_POINT = """ +(args) => { + const [panelId, fraction, padding] = args; + const [left, right, top, bottom] = padding; + const panel = window._aplHandle.api.panels.get(panelId); + const state = panel.state; + const viewFirst = state.view_x0 || 0, viewLast = state.view_x1 || 1; + const plotLeft = left, plotWidth = Math.max(1, panel.pw - left - right); + const x = plotLeft + ((fraction - viewFirst) / ((viewLast - viewFirst) || 1)) * plotWidth; + const y = top + (panel.ph - top - bottom) / 2; + const rect = panel.overlayCanvas.getBoundingClientRect(); + return [rect.left + x * (rect.width / panel.pw), + rect.top + y * (rect.height / panel.ph)]; +} +""" + + +def _movie(frames=6, height=16, width=16): + rng = np.random.default_rng(11) + return rng.integers(0, 256, size=(frames, height, width)).astype(np.uint8) + + +def _stack_page(movie, *, widget="vline"): + """A 1-D navigator (per-frame sums) driving the frame panel beside it.""" + fig, axes = apl.subplots(1, 2, figsize=(640, 300)) + times = np.arange(movie.shape[0], dtype=float) * 0.5 + navigator = axes[0].plot(movie.sum(axis=(1, 2)).astype(float), axes=[times]) + frame_plot = axes[1].imshow(movie[0], cmap="gray") + if widget == "vline": + navigator.add_vline_widget(float(times[0])) + else: + navigator.add_range_widget(float(times[0]), float(times[1])) + + html = navigated_html( + fig, {"movie": movie}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": frame_plot._id, "role": "driven", + "frame": {"block": "movie", "kind": "image", "levels": [0, 255]}}]) + return html, navigator._id, frame_plot._id + + +def _drag_1d(page, panel_id, from_fraction, to_fraction): + """Drag a 1-D widget between two positions given as axis fractions.""" + padding = [PAD_LEFT, PAD_RIGHT, PAD_TOP, PAD_BOTTOM] + x0, y0 = page.evaluate(_ONE_D_POINT, [panel_id, from_fraction, padding]) + x1, y1 = page.evaluate(_ONE_D_POINT, [panel_id, to_fraction, padding]) + page.mouse.move(x0, y0) + page.mouse.down() + page.mouse.move(x1, y1, steps=4) + page.mouse.up() + page.evaluate( + "() => new Promise(r => requestAnimationFrame(" + " () => requestAnimationFrame(() => requestAnimationFrame(r))))") + + +class TestOneDimensionalNavigator: + def test_vline_drag_paints_that_time_frame(self, navigated_page): + movie = _movie() + html, navigator_id, frame_id = _stack_page(movie) + page = navigated_page(html) + assert np.array_equal(_painted_codes(page, frame_id), movie[0].ravel()) + + _drag_1d(page, navigator_id, 0.0, 4 / (movie.shape[0] - 1)) + assert page.evaluate("() => window._aplHandle.index") == [4] + assert np.array_equal(_painted_codes(page, frame_id), movie[4].ravel()) + + def test_range_selection_gathers_the_span(self, navigated_page): + movie = _movie() + html, navigator_id, frame_id = _stack_page(movie, widget="range") + page = navigated_page(html) + + span = movie.shape[0] - 1 + _drag_1d(page, navigator_id, 1 / span, 4 / span) + indices = page.evaluate("() => window._aplHandle.index") + selected = [pair[0] for pair in indices] + assert len(selected) > 1, f"the span selected one position: {indices}" + + expected = _to_codes(movie[selected].astype(np.float64).mean(axis=0).ravel(), + 0, 255) + painted = _painted_codes(page, frame_id).astype(np.int16) + assert np.abs(painted - expected.astype(np.int16)).max() <= 1 + + +class TestStaticOverlaysSurvive: + def test_a_figure_marker_outlives_a_dispatch(self, navigated_page): + signal, spots = _dataset() + fig, axes = apl.subplots(1, 2, figsize=(640, 320)) + navigator = axes[0].imshow(signal.sum(axis=(2, 3)).astype(np.float32), cmap="gray") + signal_plot = axes[1].imshow(signal[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + signal_plot.add_circles([[10.0, 20.0]], name="fixed", radius=6, + edgecolors="#ff00ff") + + html = navigated_html( + fig, {"signal": signal, "spots": spots}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal_plot._id, "role": "driven", + "frame": {"block": "signal", "kind": "image", "levels": [0, 255]}, + "overlays": [{"block": "spots", "kind": "circles", + "style": {"radius": 4, "color": "#00ff00"}}]}]) + page = navigated_page(html) + _drag(page, navigator._id, (0, 0), (3, 3)) + + names = page.evaluate( + "(id) => window._aplHandle.api.panels.get(id).state.markers" + " .map((group) => group.name)", + signal_plot._id) + assert "fixed" in names, f"the figure's own marker group was wiped: {names}" + assert "spots" in names, f"the overlay group is missing: {names}" + + +class TestReaderGuards: + def test_ragged_index_arity_is_checked(self, navigated_page): + signal, spots = _dataset() + html, _, _ = _build_page(signal, spots, detector=False) + page = navigated_page(html) + message = page.evaluate( + """() => { + const block = window._aplHandle.blocks.spots; + const flat = {kind: 'ragged', columns: block.columns, + offsets: block.offsets, navShape: null}; + try { + window._aplModule.embed.ragged(flat).at([2, 3]); + return null; + } catch (e) { return String(e.message); } + }""") + assert message is not None, "a 2-D index into a flat block was accepted" + assert "1-D navigation grid" in message and "nav_shape" in message + + def test_reduce_checks_the_mask_covers_the_signal_grid(self, navigated_page): + signal, spots = _dataset() + html, _, _ = _build_page(signal, spots, detector=False) + page = navigated_page(html) + message = page.evaluate( + """() => { + const reader = window._aplModule.embed.dense( + window._aplHandle.blocks.signal); + try { reader.reduce(new Uint8Array(100)); return null; } + catch (e) { return String(e.message); } + }""") + assert message is not None, "a short mask was accepted" + assert "64x64 signal grid" in message + + def test_to_u8_saturates_infinity_and_zeroes_nan(self, navigated_page): + signal, spots = _dataset() + html, _, _ = _build_page(signal, spots, detector=False) + page = navigated_page(html) + codes = page.evaluate( + "() => Array.from(window._aplModule.embed.toU8(" + " [Infinity, -Infinity, NaN, 0, 1], 0, 1))") + assert codes == [255, 0, 0, 0, 255] + + +class TestViews: + def _page(self): + rng = np.random.default_rng(5) + first = rng.random((4, 4, 8, 8)).astype(np.float32) + second = (first * -1).astype(np.float32) + fig, axes = apl.subplots(1, 2, figsize=(560, 280)) + navigator = axes[0].imshow(first.sum(axis=(2, 3)), cmap="gray") + panel = axes[1].imshow(first[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + html = navigated_html( + fig, {"exx": first, "eyy": second}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "views": [{"label": "exx", "block": "exx"}, + {"label": "eyy", "block": "eyy"}], + "frame": {"kind": "image", "levels": [-1, 1]}}]) + return html, navigator._id, panel._id, first, second + + def test_the_first_view_is_shown_and_a_click_swaps_the_block(self, navigated_page): + html, navigator_id, panel_id, first, second = self._page() + page = navigated_page(html) + assert np.array_equal(_painted_codes(page, panel_id), + _to_codes(first[0, 0].ravel(), -1, 1)) + + _drag(page, navigator_id, (0, 0), (2, 1)) + assert np.array_equal(_painted_codes(page, panel_id), + _to_codes(first[1, 2].ravel(), -1, 1)) + + page.evaluate( + """async (id) => { + document.querySelector(`#apl-views-${id} button[data-view="1"]`).click(); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + panel_id) + assert np.array_equal(_painted_codes(page, panel_id), + _to_codes(second[1, 2].ravel(), -1, 1)), ( + "picking a view did not re-read the frame from its block") + pressed = page.evaluate( + f"() => [...document.querySelectorAll('#apl-views-{panel_id} button')]" + " .filter((button) => button.getAttribute('aria-pressed') === 'true')" + " .map((button) => button.textContent)") + assert pressed == ["eyy"] + + +class TestDetectorValidation: + def test_a_reduce_binding_without_a_detector_is_refused(self, _pw_browser, tmp_path): + signal, _ = _dataset() + fig, axes = apl.subplots(1, 2, figsize=(560, 280)) + navigator = axes[0].imshow(signal.sum(axis=(2, 3)).astype(np.float32)) + signal_plot = axes[1].imshow(signal[0, 0]) + navigator.add_widget("crosshair", cx=0, cy=0) + + html = navigated_html( + fig, {"signal": signal}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal_plot._id, "role": "driven", + "frame": {"block": "signal", "kind": "image"}, + "reduce": {"block": "signal", "navigator_panel": navigator._id}}]) + path = tmp_path / "no-detector.html" + path.write_text(html, encoding="utf-8") + + page = _pw_browser.new_page() + try: + page.goto(path.as_uri()) + page.wait_for_function( + "() => document.getElementById('apl-host').textContent.length > 0", + timeout=20_000) + message = page.evaluate( + "() => document.getElementById('apl-host').textContent") + finally: + page.close() + assert "carries no rectangle/circle/annular widget" in message, message + + def test_a_non_detector_widget_on_the_reduce_panel_is_ignored(self, navigated_page): + signal, _ = _dataset() + fig, axes = apl.subplots(1, 2, figsize=(640, 320)) + navigator = axes[0].imshow(signal.sum(axis=(2, 3)).astype(np.float32), cmap="gray") + signal_plot = axes[1].imshow(signal[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + signal_plot.add_widget("rectangle", x=8, y=12, w=16, h=10) + signal_plot.add_widget("crosshair", cx=40, cy=40) + + html = navigated_html( + fig, {"signal": signal}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal_plot._id, "role": "driven", + "frame": {"block": "signal", "kind": "image", "levels": [0, 255]}, + "reduce": {"block": "signal", "navigator_panel": navigator._id}}]) + page = navigated_page(html) + failures = [] + page.on("pageerror", lambda error: failures.append(str(error))) + _drag(page, signal_plot._id, (40, 40), (44, 44)) + assert failures == [], failures + + +class TestVirtualImage: + def test_a_circle_detector_gives_the_navigator_the_einsum(self, navigated_page): + signal, _ = _dataset() + fig, axes = apl.subplots(1, 2, figsize=(640, 320)) + navigator = axes[0].imshow(signal.sum(axis=(2, 3)).astype(np.float32), cmap="gray") + signal_plot = axes[1].imshow(signal[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + detector = {"type": "circle", "cx": 30.0, "cy": 34.0, "r": 11.0} + signal_plot.add_widget("circle", cx=detector["cx"], cy=detector["cy"], + r=detector["r"]) + + html = navigated_html( + fig, {"signal": signal}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal_plot._id, "role": "driven", + "frame": {"block": "signal", "kind": "image", "levels": [0, 255]}, + "reduce": {"block": "signal", "navigator_panel": navigator._id}}]) + page = navigated_page(html) + + result = page.evaluate( + """(args) => { + const [widget, height, width] = args; + const embed = window._aplModule.embed; + const mask = embed.maskFromWidget(widget, width, height); + const reader = embed.dense(window._aplHandle.blocks.signal); + return {mask: Array.from(mask), values: Array.from(reader.reduce(mask))}; + }""", + [detector, SIGNAL_SHAPE[0], SIGNAL_SHAPE[1]]) + + mask = np.asarray(result["mask"], dtype=np.uint8).reshape(SIGNAL_SHAPE) + assert mask.sum() > 0 + expected = np.einsum("...ij,ij->...", signal.astype(np.float64), mask).ravel() + assert np.array_equal(np.asarray(result["values"], dtype=np.float64), expected) + + +class TestGeomPushKeepsTheLiveFrame: + def test_a_geom_push_leaves_the_binary_token_in_place(self, navigated_page): + signal, spots = _dataset() + html, _, signal_id = _build_page(signal, spots, detector=False) + page = navigated_page(html) + + pushed = page.evaluate( + """async (panelId) => { + const handle = window._aplHandle; + handle.setImage(panelId, new Uint8Array(64 * 64).fill(66), 64, 64, + {display_min: 0, display_max: 255}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + // A host pushing the panel's geometry again (a colormap change, + // say) must not rename the key that identifies the live bytes. + const geom = handle.get(`panel_${panelId}_geom`); + handle.applyUpdate(`panel_${panelId}_geom`, geom); + await new Promise((r) => requestAnimationFrame(r)); + const panel = handle.api.panels.get(panelId); + return {token: panel._geomCache.image_b64, + codes: Array.from(panel.state.image_b64_bytes.slice(0, 4))}; + }""", + signal_id) + assert pushed["token"].startswith(BINARY_TOKEN_PREFIX), ( + f"a geom push renamed the live binary frame to {pushed['token'][:24]!r}") + assert pushed["codes"] == [66, 66, 66, 66] + + def test_a_geom_push_carrying_its_own_token_wins(self, navigated_page): + signal, spots = _dataset() + html, _, signal_id = _build_page(signal, spots, detector=False) + page = navigated_page(html) + token = page.evaluate( + """async (panelId) => { + const handle = window._aplHandle; + handle.setImage(panelId, new Uint8Array(64 * 64).fill(9), 64, 64); + await new Promise((r) => requestAnimationFrame(r)); + const geom = JSON.parse(handle.get(`panel_${panelId}_geom`)); + geom.image_b64 = '\\u0000bin:99999'; + handle.applyUpdate(`panel_${panelId}_geom`, JSON.stringify(geom)); + await new Promise((r) => requestAnimationFrame(r)); + return handle.api.panels.get(panelId)._geomCache.image_b64; + }""", + signal_id) + assert token == BINARY_TOKEN_PREFIX + "99999" + + +class TestTwoFiguresInOneDocument: + def test_two_mounts_keep_their_own_pixels(self, navigated_page): + """Panel ids hash the layout position, so two identical figures in one + document write the same global pixel slot.""" + signal, spots = _dataset() + html, _, signal_id = _build_page(signal, spots, detector=False) + page = navigated_page(html) + + page.evaluate( + """async () => { + const host = document.createElement('div'); + document.body.appendChild(host); + window._secondHandle = await window._aplModule.mountNavigated( + host, window._aplPage, {}); + }""") + page.wait_for_function("() => window._secondHandle !== undefined", timeout=20_000) + + result = page.evaluate( + """async (panelId) => { + const read = (handle) => { + const panel = handle.api.panels.get(panelId); + return [Array.from(panel.state.image_b64_bytes.slice(0, 2)), + panel._geomCache.image_b64]; + }; + window._aplHandle.setImage(panelId, new Uint8Array(64 * 64).fill(11), + 64, 64, {display_min: 0, display_max: 255}); + window._secondHandle.setImage(panelId, new Uint8Array(64 * 64).fill(222), + 64, 64, {display_min: 0, display_max: 255}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + return {first: read(window._aplHandle), second: read(window._secondHandle)}; + }""", + signal_id) + assert result["first"][0] == [11, 11], result + assert result["second"][0] == [222, 222], result + assert result["first"][1] != result["second"][1], ( + f"both figures claimed the pixel key {result['first'][1]!r}") + + +class TestFrameLevels: + def test_the_panel_window_is_kept_across_positions(self, navigated_page): + """Without fixed levels the window is the panel's, not a per-frame one.""" + rng = np.random.default_rng(2) + block = (rng.random((4, 4, 8, 8)) * 100).astype(np.float32) + block[2, 2] += 900.0 # one very bright position + fig, axes = apl.subplots(1, 2, figsize=(560, 280)) + navigator = axes[0].imshow(block.sum(axis=(2, 3)), cmap="gray") + panel = axes[1].imshow(block[0, 0], cmap="gray", vmin=0.0, vmax=100.0) + navigator.add_widget("crosshair", cx=0, cy=0) + html = navigated_html( + fig, {"cube": block}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": panel._id, "role": "driven", + "frame": {"block": "cube", "kind": "image"}}]) + page = navigated_page(html) + _drag(page, navigator._id, (0, 0), (2, 2)) + + window = page.evaluate( + "(id) => { const s = window._aplHandle.api.panels.get(id).state;" + " return [s.display_min, s.display_max]; }", + panel._id) + assert window == [0.0, 100.0], ( + f"the frame window drifted to {window} instead of keeping the panel's") + assert np.array_equal(_painted_codes(page, panel._id), + _to_codes(block[2, 2].ravel(), 0.0, 100.0)) + + +class TestGeometryLandsWithItsBytes: + def test_a_resize_is_not_applied_before_the_bytes(self, navigated_page): + signal, spots = _dataset() + html, _, signal_id = _build_page(signal, spots, detector=False) + page = navigated_page(html) + sizes = page.evaluate( + """async (panelId) => { + const handle = window._aplHandle; + const state = () => handle.api.panels.get(panelId).state; + handle.setImage(panelId, new Uint8Array(16 * 32).fill(5), 32, 16, + {display_min: 0, display_max: 255}); + const beforeFrame = [state().image_width, state().image_height]; + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + const afterFrame = [state().image_width, state().image_height]; + return {beforeFrame, afterFrame, + bytes: state().image_b64_bytes.length}; + }""", + signal_id) + assert sizes["beforeFrame"] == [64, 64], ( + "the panel was resized before the bytes it describes were committed") + assert sizes["afterFrame"] == [32, 16] + assert sizes["bytes"] == 32 * 16 + + +class TestOverlayStylePassesThrough: + def test_fill_alpha_reaches_the_wire_and_paints_opaque(self, navigated_page): + """An allow-list here silently drops whatever it has not heard of.""" + signal, spots = _dataset() + fig, axes = apl.subplots(1, 2, figsize=(640, 320)) + navigator = axes[0].imshow(signal.sum(axis=(2, 3)).astype(np.float32), cmap="gray") + signal_plot = axes[1].imshow(np.zeros(SIGNAL_SHAPE, dtype=np.uint8), cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + + style = {"radius": 9, "color": "#ff0000", "fill_color": "#00ff00", + "fill_alpha": 1.0} + html = navigated_html( + fig, {"signal": signal, "spots": spots}, + [{"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal_plot._id, "role": "driven", + "frame": {"block": "signal", "kind": "image", "levels": [0, 255]}, + "overlays": [{"block": "spots", "kind": "circles", "style": style}]}]) + page = navigated_page(html) + + wire = page.evaluate( + "(id) => window._aplHandle.api.panels.get(id).state.markers[0]", + signal_plot._id) + assert wire["fill_alpha"] == 1.0, wire + assert wire["fill_color"] == "#00ff00", wire + assert "radius" not in wire, "the sizes input leaked into the wire dict" + assert wire["sizes"] == [9] * ROWS_PER_POSITION + + # A fully opaque fill paints the fill colour, not a 30 % blend of it. + opaque = page.evaluate( + """(id) => { + const canvas = window._aplHandle.api.panels.get(id).markersCanvas; + const data = canvas.getContext('2d') + .getImageData(0, 0, canvas.width, canvas.height).data; + let n = 0; + for (let i = 0; i < data.length; i += 4) + if (data[i] < 40 && data[i + 1] > 215 && data[i + 2] < 40 + && data[i + 3] > 250) n++; + return n; + }""", + signal_plot._id) + assert opaque > 20, f"only {opaque} fully opaque green pixels" diff --git a/anyplotlib/tests/test_embed/test_embed_set_image.py b/anyplotlib/tests/test_embed/test_embed_set_image.py new file mode 100644 index 00000000..16f7040a --- /dev/null +++ b/anyplotlib/tests/test_embed/test_embed_set_image.py @@ -0,0 +1,306 @@ +""" +Playwright tests for ``handle.setImage``, the binary image setter. + +A navigated page scrubs frames as fast as the user drags, so the cost the +caller pays to hand one over has to be independent of the frame size. These +tests pin that: the push is a side-table write plus one animation-frame +request, and the paint it schedules is the renderer's ordinary blit. +""" +from __future__ import annotations + +import numpy as np +import pytest + +import anyplotlib as apl + +# A frame push must stay well inside one 60 Hz frame even at 2048², where the +# base64 route cost 129-136 ms of main-thread time. +PUSH_BUDGET_MS = 5.0 + +# One measured push per frame, enough samples for a stable median. +FRAME_COUNT = 40 + + +def _figure_with_panel(): + """A small grayscale panel; setImage grows it to the size under test.""" + fig, ax = apl.subplots(1, 1, figsize=(320, 320)) + plot = ax.imshow(np.zeros((16, 16), dtype=np.uint8), cmap="gray") + return fig, plot + + +_PUSH_SCRIPT = """ +async (args) => { + const [panelId, size, frameCount] = args; + const handle = window._handle; + const pixels = size * size; + // Eight distinct frames, cycled: enough to prove the canvas follows the + // pushes without holding forty full-size buffers alive at once. + const frames = []; + for (let f = 0; f < 8; f++) { + const frame = new Uint8Array(pixels); + for (let i = 0; i < pixels; i++) frame[i] = (i + f * 31) & 255; + frames.push(frame); + } + const nextFrame = () => new Promise((r) => requestAnimationFrame(r)); + const durations = []; + for (let n = 0; n < frameCount; n++) { + const frame = frames[n % frames.length]; + const start = performance.now(); + handle.setImage(panelId, frame, size, size, {display_min: 0, display_max: 255}); + durations.push(performance.now() - start); + await nextFrame(); + await nextFrame(); + } + // performance.now() is clamped to 100 us in a page that is not + // cross-origin-isolated, so one push lands on 0.0 or 0.1 and the median says + // little. Timing the whole run of pushes back to back resolves it; they + // coalesce into one paint, which is the steady-state scrub cost. + const batchCount = frameCount * 25; + const batchStart = performance.now(); + for (let n = 0; n < batchCount; n++) + handle.setImage(panelId, frames[n % frames.length], size, size, + {display_min: 0, display_max: 255}); + const perPush = (performance.now() - batchStart) / batchCount; + await nextFrame(); + await nextFrame(); + + // The first push carries the panel resize, so it is reported on its own. + const first = durations[0]; + const rest = durations.slice(1).sort((a, b) => a - b); + return {first, perPush, + median: rest[rest.length >> 1], worst: rest[rest.length - 1]}; +} +""" + +_SAMPLE_SCRIPT = """ +(args) => { + const [panelId, ix, iy] = args; + const panel = window._handle.api.panels.get(panelId); + const state = panel.state; + const scale = Math.min(panel.imgW / state.image_width, + panel.imgH / state.image_height); + const fitWidth = state.image_width * scale, fitHeight = state.image_height * scale; + const left = (panel.imgW - fitWidth) / 2, top = (panel.imgH - fitHeight) / 2; + const device = panel.plotCanvas.width / panel.imgW; + const x = Math.floor((left + (ix + 0.5) / state.image_width * fitWidth) * device); + const y = Math.floor((top + (iy + 0.5) / state.image_height * fitHeight) * device); + const data = panel.plotCanvas.getContext('2d').getImageData(x, y, 1, 1).data; + return [data[0], data[1], data[2]]; +} +""" + + +def _sample(page, panel_id, ix, iy): + return page.evaluate(_SAMPLE_SCRIPT, [panel_id, ix, iy]) + + +def _colormap_rgb(page, panel_id, code): + """The colour the renderer's lookup table gives one 8-bit code.""" + return page.evaluate( + "(args) => window._handle.api.panels.get(args[0])" + " .state.colormap_data[args[1]]", + [panel_id, code]) + + +class TestSetImageCost: + @pytest.mark.parametrize("size", [512, 2048]) + def test_push_median_is_independent_of_frame_size(self, mount_page, size, capsys): + fig, plot = _figure_with_panel() + page = mount_page(fig) + timings = page.evaluate(_PUSH_SCRIPT, [plot._id, size, FRAME_COUNT]) + with capsys.disabled(): + print(f"\nsetImage {size}x{size} over {FRAME_COUNT} frames: " + f"median {timings['median']:.3f} ms, worst {timings['worst']:.3f} ms, " + f"{timings['perPush']:.4f} ms per push back to back, " + f"first (carries the resize) {timings['first']:.3f} ms") + assert timings["median"] < PUSH_BUDGET_MS, ( + f"setImage median {timings['median']:.3f} ms at {size}² exceeds " + f"the {PUSH_BUDGET_MS} ms budget") + # The back-to-back mean is the one number the clock resolves properly, + # and it is where a per-frame blit would show up as a regression. + assert timings["perPush"] < PUSH_BUDGET_MS, ( + f"setImage costs {timings['perPush']:.4f} ms per push at {size}², " + f"over the {PUSH_BUDGET_MS} ms budget") + + +class TestSetImagePaints: + def test_pushed_frames_reach_the_canvas(self, mount_page): + fig, plot = _figure_with_panel() + page = mount_page(fig) + + def push(fill): + page.evaluate( + """async (args) => { + const [panelId, fill] = args; + const frame = new Uint8Array(64 * 64).fill(fill); + window._handle.setImage(panelId, frame, 64, 64, + {display_min: 0, display_max: 255}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + [plot._id, fill]) + + push(40) + dark = _sample(page, plot._id, 32, 32) + push(210) + bright = _sample(page, plot._id, 32, 32) + assert dark == _colormap_rgb(page, plot._id, 40), ( + f"pushed code 40 painted as {dark}") + assert bright == _colormap_rgb(page, plot._id, 210), ( + f"pushed code 210 painted as {bright}") + + def test_geometry_follows_the_bytes(self, mount_page): + """A frame of a different size repaints at that size, not the old one.""" + fig, plot = _figure_with_panel() + page = mount_page(fig) + page.evaluate( + """async (panelId) => { + const frame = new Uint8Array(32 * 8).fill(100); + window._handle.setImage(panelId, frame, 32, 8); + await new Promise((r) => requestAnimationFrame(r)); + }""", + plot._id) + geometry = page.evaluate( + "(id) => { const s = window._handle.api.panels.get(id).state;" + " return [s.image_width, s.image_height, s.base_width]; }", + plot._id) + assert geometry == [32, 8, 0] + + def test_length_mismatch_throws(self, mount_page): + fig, plot = _figure_with_panel() + page = mount_page(fig) + message = page.evaluate( + """(panelId) => { + try { + window._handle.setImage(panelId, new Uint8Array(10), 8, 8); + return null; + } catch (e) { return String(e.message); } + }""", + plot._id) + assert message is not None, "a short buffer was accepted" + assert "64 bytes" in message and "got 10" in message + + def test_unknown_panel_throws(self, mount_page): + fig, _ = _figure_with_panel() + page = mount_page(fig) + message = page.evaluate( + """() => { + try { + window._handle.setImage('nope', new Uint8Array(4), 2, 2); + return null; + } catch (e) { return String(e.message); } + }""") + assert message is not None and "unknown panel" in message + + def test_no_sync_echo(self, mount_page): + """The pixels are an inbound update; they must not bounce to onSync.""" + fig, plot = _figure_with_panel() + page = mount_page(fig) + page.evaluate("() => { window._syncs = []; }") + page.evaluate( + """async (panelId) => { + window._handle.setImage(panelId, new Uint8Array(64 * 64).fill(7), + 64, 64, {display_min: 0, display_max: 255}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + plot._id) + keys = page.evaluate("() => (window._syncs || []).map((s) => s.key)") + assert keys == [], f"setImage echoed through onSync: {keys}" + + +class TestSetImageTargets: + def test_a_non_image_panel_is_refused(self, mount_page): + """A 3-D panel has a geometry trait but nowhere to put pixel bytes.""" + fig, ax = apl.subplots(1, 1, figsize=(320, 320)) + grid = np.linspace(-1.0, 1.0, 8) + x, y = np.meshgrid(grid, grid) + surface = ax.plot_surface(x, y, (x ** 2 + y ** 2).astype(np.float32)) + page = mount_page(fig) + message = page.evaluate( + """(panelId) => { + try { + window._handle.setImage(panelId, new Uint8Array(64), 8, 8); + return null; + } catch (e) { return String(e.message); } + }""", + surface._id) + assert message is not None, "a 3-D panel accepted pixel bytes" + assert "only a 2-D image panel" in message + + def test_a_new_frame_voids_the_detail_tile(self, mount_page): + """A detail tile crops the PREVIOUS frame, so it cannot outlive it.""" + rng = np.random.default_rng(4) + base = rng.integers(0, 256, size=(1200, 1200)).astype(np.uint8) + fig, ax = apl.subplots(1, 1, figsize=(320, 320)) + plot = ax.imshow(base, cmap="gray", tile=True) + plot.set_detail(base[0:128, 0:128], x0=0, x1=128, y0=0, y1=128) + assert plot.to_state_dict()["detail_width"] > 0, "no detail tile to clear" + + page = mount_page(fig) + before = page.evaluate( + "(id) => { const p = window._handle.api.panels.get(id);" + " return [p.state.detail_width, !!p._geomCache.detail_b64," + " !!p.state.detail_b64_bytes]; }", + plot._id) + assert before[0] > 0 and (before[1] or before[2]), before + + page.evaluate( + """async (panelId) => { + window._handle.setImage(panelId, new Uint8Array(64 * 64).fill(80), + 64, 64, {display_min: 0, display_max: 255}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + plot._id) + after = page.evaluate( + """(id) => { + const p = window._handle.api.panels.get(id); + return {width: p.state.detail_width, height: p.state.detail_height, + region: p.state.detail_region, + geom: p._geomCache.detail_b64, + geomBytes: p._geomCache.detail_b64_bytes === undefined, + stateBytes: p.state.detail_b64_bytes === undefined, + tile: p.state.tile_enabled, base: p.state.base_width}; + }""", + plot._id) + assert after["width"] == 0 and after["height"] == 0, after + assert after["region"] == [], after + assert after["geom"] == "" and after["geomBytes"], after + assert after["stateBytes"], after + assert after["tile"] is False and after["base"] == 0, after + + def test_rgba_bytes_paint_their_own_channels(self, mount_page): + fig, ax = apl.subplots(1, 1, figsize=(320, 320)) + plot = ax.imshow(np.zeros((16, 16), dtype=np.uint8), cmap="gray") + page = mount_page(fig) + page.evaluate( + """async (panelId) => { + const size = 32, pixels = size * size; + const frame = new Uint8Array(pixels * 4); + for (let i = 0; i < pixels; i++) { + frame[i * 4] = 200; frame[i * 4 + 1] = 30; + frame[i * 4 + 2] = 90; frame[i * 4 + 3] = 255; + } + window._handle.setImage(panelId, frame, size, size, {rgb: true}); + await new Promise((r) => requestAnimationFrame(r)); + await new Promise((r) => requestAnimationFrame(r)); + }""", + plot._id) + assert page.evaluate( + "(id) => window._handle.api.panels.get(id).state.is_rgb", plot._id) is True + assert _sample(page, plot._id, 16, 16) == [200, 30, 90] + + def test_rgba_length_is_four_bytes_a_pixel(self, mount_page): + fig, plot = _figure_with_panel() + page = mount_page(fig) + message = page.evaluate( + """(panelId) => { + try { + window._handle.setImage(panelId, new Uint8Array(8 * 8 * 3), 8, 8, + {rgb: true}); + return null; + } catch (e) { return String(e.message); } + }""", + plot._id) + assert message is not None and "256 bytes for 8x8 RGBA" in message diff --git a/docs/embedding.rst b/docs/embedding.rst index 42f672ce..62d66379 100644 --- a/docs/embedding.rst +++ b/docs/embedding.rst @@ -143,6 +143,180 @@ changes — streams to the window automatically; drags, clicks, and keys stream back into your Python callbacks. Echo is suppressed in both directions by the bridge and ``applyUpdate``. +Navigated pages — one page that owns its data +============================================== + +A *navigated* figure is one where a navigator panel drives the others: move the +crosshair over the scan and the signal panel shows that position's frame, its +overlays follow, and a detector drawn on the signal panel reduces the whole +dataset back onto the navigator. :func:`~anyplotlib.embed.navigated_html` +exports that as a single file — the renderer, the figure state, the data and the +bindings all inlined, no network and no Python at view time. + +The data travels as *blocks*. A **dense** block is a numpy array whose leading +axes are the navigation axes; a :class:`~anyplotlib.embed.Ragged` block is a +row-pointer array plus one value array per column, for a variable number of rows +per position (diffraction spots, detected particles, peaks). +:func:`~anyplotlib.embed.pack_blocks` concatenates them into one little-endian +byte string, which the page decodes once into a single ``ArrayBuffer`` and reads +through typed-array views — no per-block base64, no copy per frame. + +:: + + import numpy as np + import anyplotlib as apl + from anyplotlib.embed import navigated_html + + scan = np.load("scan.npy") # (32, 32, 128, 128) uint8 + + fig, axes = apl.subplots(1, 2, figsize=(760, 380)) + navigator = axes[0].imshow(scan.sum(axis=(2, 3)), cmap="gray") + signal = axes[1].imshow(scan[0, 0], cmap="gray") + navigator.add_widget("crosshair", cx=0, cy=0) + signal.add_widget("rectangle", x=48, y=48, w=32, h=32) # the detector + + html = navigated_html( + fig, + {"scan": scan}, + [ + {"panel_id": navigator._id, "role": "navigator"}, + {"panel_id": signal._id, "role": "driven", + "frame": {"block": "scan", "kind": "image"}, + "reduce": {"block": "scan", "navigator_panel": navigator._id}}, + ], + title="Scan", caption="Drag the crosshair; drag the detector to re-map.", + ) + open("scan.html", "w", encoding="utf-8").write(html) + +Open ``scan.html`` in any browser, or point an Electron window or an ``