Navigated embeds: a page that owns its data, and a binary image setter on the mount handle - #69
Merged
Conversation
The standalone page's postMessage export handler moves out of the page template into PNG_HARVEST_LISTENER, and reads its export entry point from globalThis.__aplExportPNG instead of the template's own render() api, so a second kind of page can install the same listener without copying it.
setImage hands a 2-D panel raw pixel bytes instead of routing a frame through the geom trait: measured on 0.8.0, base64 costs 6.7 ms at 512 square and 129-136 ms at 2048 square of main-thread time, against a fraction of a millisecond here. Two details are load-bearing. The panel's geom cache gets a fresh image_b64 token per push, because the blit cache keys on that string and a mount() page's geom carries real base64 that never changes. And the repaint is coalesced onto the next animation frame, so a task that pushes several frames pays one blit; exportPNG flushes first. patchPanel and panelIds round the handle out. Above that sits mountNavigated: a page hands over blocks of data plus bindings saying which block feeds which panel, and a navigator widget's position dispatches through readers (dense, ragged, maskFromWidget, rasterDisks, robustLevels, toU8) onto every driven panel. A detector widget on a driven panel runs the other way, reducing the block back onto the navigator's image. FIGURE_ESM.md gains a section for it, and every line anchor in both numbered tables is regenerated from the file (they had drifted by 37 to 100 lines before this change).
pack_blocks packs dense arrays and Ragged row-pointer blocks into one little-endian byte string plus a manifest, aligned so every typed-array view is legal; the page decodes it once and takes views, rather than paying base64 per block. navigated_html inlines the renderer, the figure state, that blob and the bindings into a self-contained page, and refuses a binding that names a panel or a block the page does not carry.
test_embed_navigated.py opens real navigated_html output from file:// and drags: the dragged index has to paint that frame (bytes and canvas pixels alike), a ragged circles overlay has to land on its rows, a detector rectangle has to reduce to numpy's einsum, and a rectangle navigator selection has to paint the gathered mean. rasterDisks, maskFromWidget and robustLevels are checked against numpy references. test_embed_set_image.py pins the push cost at 512 and 2048 square and prints the medians, and covers painting, the geometry patch, both throws, and the absence of an onSync echo. The shared mount page now records onSync writes so that last one can see them.
docs/embedding.rst gains a "Navigated pages" section with a complete example (numpy block to pack_blocks to navigated_html to a file), the binding table, and the JS entry points; the handle reference lists patchPanel, setImage, flushImages and panelIds.
A drag fires an event per pointer move and each reduce is a pass over the whole block, so the detector path gets the same newest-wins animation-frame gate the navigator dispatch already had. Also drops the em-dashes from the new comments and refreshes the recorded file length.
decodeBlocks takes an empty ArrayBuffer rather than fetching an empty data URL, so a page with no blocks mounts; embed.py imports numpy at module scope for the Ragged annotation.
Only membership was ever read, so it is a set rather than a map of sizes.
performance.now() is clamped to 100 microseconds outside a cross-origin-isolated page, so a single push reads 0.0 or 0.1 ms and the median says nothing. Timing a thousand pushes back to back gives the real steady-state number, and that is what the budget now asserts; the per-frame median and worst are still printed.
It no longer sits under the state-update handler it used to point back at.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #69 +/- ##
==========================================
+ Coverage 90.97% 91.08% +0.10%
==========================================
Files 41 41
Lines 4709 4789 +80
==========================================
+ Hits 4284 4362 +78
- Misses 425 427 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved security and renderer/API correctness issues remain.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds self-contained navigated HTML embeds and efficient binary image updates for interactive plots.
Changes:
- Adds packed data blocks and
mountNavigated. - Adds image, panel patching, flushing, and navigation APIs.
- Updates documentation, tests, changelog, and renderer references.
File summaries
| File | Reviewed changes |
|---|---|
upcoming_changes/69.new_feature.rst |
Changelog entry for navigated embeds and binary image updates |
docs/embedding.rst |
Navigated embedding and API documentation |
anyplotlib/tests/test_embed/test_embed_set_image.py |
Binary image setter tests |
anyplotlib/tests/test_embed/test_embed_navigated.py |
End-to-end navigated embed tests |
anyplotlib/tests/test_embed/test_embed_api.py |
Packing and HTML API tests |
anyplotlib/tests/test_embed/_export_utils.py |
Mount synchronization test helpers |
anyplotlib/FIGURE_ESM.md |
Updated renderer section references |
anyplotlib/figure_esm.js |
Mount APIs, image updates, and navigation runtime |
anyplotlib/embed.py |
Data block packing and navigated HTML generation |
anyplotlib/_repr_utils.py |
Shared PNG harvest listener |
AGENTS.md |
Updated renderer size references |
Review details
Suppressed comments (3)
anyplotlib/figure_esm.js:11733
- Plot1D serializes its x axis as
x_axis_b64and the rawstate.x_axisis absent (seeanyplotlib/plot1d/_plot1d.py:381-387). Consequentlystate.x_axisis undefined here and every 1-D navigatorvline/pointresolves to index 0, regardless of the widget's x position; use the panel's decoded_1dXArr(or decodex_axis_b64) for nearest-axis lookup.
if (widget.type === 'vline') return [nearestAxisIndex(state.x_axis, widget.x)];
if (widget.type === 'point') return [nearestAxisIndex(state.x_axis, widget.x)];
anyplotlib/figure_esm.js:11215
- When display limits, dimensions, or RGB mode change,
patchPanelruns synchronously and its panel listener immediately calls_redrawPanel; only the byte-slot commit is deferred.mountNavigatedcomputes robust levels per frame, so normal scrubbing can synchronously blit the previous frame before every queued raw-byte paint, defeating the low-latencysetImagepath. Defer the metadata patch/redraw together with the pending image and apply both once incommitImages.
if (Object.keys(patch).length) this.patchPanel(panelId, patch);
anyplotlib/tests/test_embed/test_embed_set_image.py:111
- This test suite covers only colormap-code frames; none of the tests calls
setImagewithopts.rgb: true. The RGBA path is a separate byte-count and renderer branch, so it can regress while all current tests remain green; add a mount-level test that pushes four-byte RGBA data and verifies the canvas pixel.
def test_push_median_is_independent_of_frame_size(self, mount_page, size, capsys):
fig, plot = _figure_with_panel()
page = mount_page(fig)
timings = page.evaluate(_PUSH_SCRIPT, [plot._id, size, FRAME_COUNT])
- Files reviewed: 11/11 changed files
- Comments generated: 6
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| caption_html=(f'<div class="apl-caption">{escape(caption)}</div>' | ||
| if caption else ""), | ||
| strips_html=strips, | ||
| page_json=_json.dumps(page, default=str), |
Comment on lines
+11185
to
+11186
| if (model.get(`panel_${panelId}_geom`) === undefined) | ||
| throw new Error(`setImage: panel ${panelId} has no image channel`); |
| if (current.base_width) patch.base_width = 0; | ||
| if (current.base_height) patch.base_height = 0; | ||
| if (current.tile_enabled) patch.tile_enabled = false; | ||
| if (current.detail_b64) patch.detail_b64 = ''; |
| const wire = { id: `apl-overlay-${overlay.block}`, name: overlay.block, | ||
| type: overlay.kind, color: style.color || '#ff0000', | ||
| linewidth: style.linewidth === undefined ? 1.5 : style.linewidth }; | ||
| if (style.fill_color) wire.fill_color = style.fill_color; |
Comment on lines
+11847
to
+11852
| function paintChips(binding, indices) { | ||
| const parts = binding.chips.map((chip) => { | ||
| const reader = readerFor(chip.block); | ||
| const values = reader.at(indices[0]); | ||
| const value = values && values.length ? values[0] : NaN; | ||
| return `${chip.label} ${Number.isFinite(value) ? value.toPrecision(4) : '-'}`; |
Comment on lines
+11188
to
+11192
| const rgb = !!settings.rgb; | ||
| const expected = width * height * (rgb ? 4 : 1); | ||
| if (!bytes || bytes.length !== expected) | ||
| throw new Error(`setImage: expected ${expected} bytes for ${width}x${height}` + | ||
| `${rgb ? ' RGBA' : ''}, got ${bytes ? bytes.length : 0}`); |
A 1-D navigator never moved: Plot1D.to_state_dict pops x_axis and sends x_axis_b64, so resolving a vline through state.x_axis read an empty array and every position came out 0. panelAxis reads draw1d's decoded cache the way the rest of the file does. The span selector, the 1-D analogue of the rectangle, was ignored in silence; it now resolves both edges and selects the run. A dispatch assigned markers and extra_lines wholesale, wiping the groups the figure was built with on the first crosshair move; it now merges by the apl-overlay- id prefix. A ragged index of the wrong arity silently read the row number as the position, and dense.reduce trusted the mask's length; both throw. views replace chips: a committed result's alternative frames, picked from the page's segmented control, rather than per-position scalars readout already covers. setImage patched geometry at push time but committed bytes on the next animation frame, so a size change painted once at the new dimensions over the old pixels; both now land in commitImages. Its sequence counter is global, because the pixel side table is and panel ids repeat across figures of one layout. _loadGeom keeps a live binary token rather than letting a geom push rename the key for bytes that are still what gets drawn. The frame's colour window defaults to the panel's own instead of a percentile recomputed per frame, which cost two passes and made the contrast jump between neighbouring positions. A reduce binding with no detector on its panel is refused at mount, and any other widget there is ignored at dispatch rather than throwing inside an animation frame. The generic readers move under one embed namespace so dense, ragged and toU8 stop sitting beside mount and render.
A views binding renders a segmented control of its labels, which the runtime wires to the block each button names. frame.block is optional when views is given. The page also publishes the dict it was built from, so a host can mount a second figure from it or read back what this one carries. json is imported at module scope rather than aliased inside the builder.
A 1-D navigator drag lands on its time frame and a span gathers the run between its edges; a figure's own marker group survives a dispatch; a mismatched ragged index and a short reduce mask both throw; a views button re-reads the frame from its block; a reduce binding with no detector is refused at mount while a crosshair on that panel is ignored at dispatch; a circle detector gives the navigator numpy's einsum; a geom push leaves a live binary token alone but yields to one of its own; two figures in one document keep their own pixels; the frame window stays the panel's; and a resize does not reach the panel before the bytes it describes. The region mean is compared with a tolerance of one code, since a float32 sum scaled by 1/n and numpy's float64 mean can truncate either side of a boundary, and the rasterDisks reference states the convention (a filled disk per row, combined by max or sum) rather than transcribing the loop.
The detector widget becomes a mask over the signal grid, every position's frame is summed under it, and the navigator-shaped result goes back with setImage. The binding table documents views, initial_index and the disk raster's width/height, and the changelog fragment loses its em dash.
An HTML parser ends a script element at the first "</" in its text and treats "<!--" as a comment opener, and it does not care that the sequence sits inside a JavaScript string. A figure title, an axis label read from file metadata, a widget field or a binding's style value carrying "</script>" therefore closed the block and ran whatever followed, in a file people open directly rather than only inside a sandboxed frame. script_json escapes both sequences through the "<", which JSON spells \u003c, so JSON.parse and a script literal read back the character that went in. Every JSON literal either page embeds now goes through it: the navigated page's state, bindings, manifest and chrome, and the standalone page's state, inlined renderer and figure id. Panel ids reaching element ids are escaped like the title and caption already were.
setImage took any panel with a geometry trait, so a 3-D panel accepted bytes that went nowhere; it now checks the panel kind. A detail tile is a crop of the PREVIOUS frame at a zoom the viewer may still be at, and clearing only the light field left the bytes and the region in the geom cache for _applyGeom to splice straight back in, so a new frame now voids the tile everywhere it lives. The circle wire builder read the style key by key and dropped fill_alpha, so every filled overlay got the renderer's 0.3 and a requested 0 or 1 was ignored. The style IS the wire dict, so it is carried through whole.
A page whose panel title, axis label, page title, caption and binding style all carry "</script><script>window.__injected = 1</script>" opens with __injected undefined, the figure mounted and one script element; the title and caption render as text. script_json is checked on its own for a literal with no "</" that json.loads reads back. The standalone page gets the same treatment. Also: a 3-D panel refuses pixel bytes; a new frame clears a tile-enabled panel's detail fields in the state, the geom cache and the side table; an overlay's fill_alpha of 1 reaches the wire and paints opaque; and RGBA bytes paint their own channels, with the wrong length refused.
Owner
Author
|
Thanks — all six are addressed.
Whole suite green. |
An orientation map's sphere is a driven panel: the picked orientation is a highlight on it, the sphere turns to face that point, and a direction toggle swaps the cloud and its per-point colours. setImage is still the wrong door for that, so the 3-D path goes through patchPanel and the geometry trait. A frame of kind points3d loads a dense (M, 3) float32 cloud with a (M, 3) or (M, 4) uint8 colors block. It rides panel_<id>_geom as base64, because the binary side table is registered for image pixel keys only and a cloud changes on a view click rather than per navigator move; vertices_count and a bumped _geom_rev go on the light trait in the same breath, or the renderer draws the previous cloud's point count. The push is skipped while the block and colours are the ones already shown, so a navigator drag costs nothing. A highlight overlay writes the shape Plot3D.set_highlight writes, from a dense (nav..., 3) or one-row-per-position ragged block. With face_camera it also writes elevation = asin(z/r) and azimuth = atan2(x, -y) plus _view_from_python, which is what tells _preserveView the camera is intended. The flag is written either way: it persists on the trait, so a true left by an earlier facing push would let the next highlight discard the orbit the reader is holding. A views entry is now identified by its position rather than its block, since two views may read one cloud with different colours.
frame.colors and views[].colors are checked like every other block name, and each view button carries its index rather than its block so a direction toggle over one cloud can tell its entries apart.
A real scatter3d panel with a reference sphere: a crosshair drag puts the block's row in the state's highlight and moves the white pixels; face_camera lands the camera on asin(z)/atan2(x, -y) and without it an orbit the reader made survives the next dispatch; a view click repaints the cloud in its own colours while a navigator move does not re-push it; the cloud arrives on the geometry trait with the right vertices and count; setImage still refuses the panel; and dense.at on a (nav, nav, 3) block gives the 3-vector. The docs gain rows for points3d, highlight and face_camera, and a paragraph on how an orientation map flows through a navigated page.
Owner
Author
|
One more capability on top of the Copilot round: a 3-D panel can now be a driven panel of a navigated page. Refusing
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Why
How a virtual image flows through it
frame: {block: "scan"}(its image at the crosshair position isscan[iy, ix]) andreduce: {block: "scan", navigator_panel: <the navigator>}.setImage. That iseinsum("...ij,ij->...", scan, mask), the same thing SpyDE's virtual image action does per chunk, done once in the browser on the data that shipped.(x, y)falls inside the mask, per position. No stack needed.How a strain map flows through it
views, a segmented control that swaps which block the panel reads; the colorbar label comes from the figure state as it does today.(x, y, u, v)rows per scan position, evaluated once in Python at export, and drawn as ordinary arrow markers at the crosshair position.What is in the diff
handle.setImage(panelId, bytes, width, height, opts): raw pixel bytes straight into the draw path, through the same side table the standalone page already used for the Electron binary transport. Colormap codes, or RGBA withopts.rgb.handle.patchPanel(panelId, partial),handle.panelIds(),handle.flushImages().mountNavigated(el, page)and the readers under it:dense/ragged(at,gather,reduce),maskFromWidget,rasterDisks,robustLevels,toU8.embed.Ragged,embed.pack_blocks(one byte string, one manifest, oneArrayBufferon the page),embed.navigated_html. A binding that names a panel or block the page does not carry raises at build time.docs/embedding.rst"Navigated pages" with a complete example.FIGURE_ESM.mdanchors regenerated (they were already 37 to 100 lines stale).Numbers
setImageis a fraction of a millisecond at either size and paints on the next animation frame. The test prints the medians.Things to know before reading the diff
ImageData.setImagealso writes a fresh\0bin:<n>token into the panel's geom cache, because the blit cache keys onimage_b64first and only falls back to the arrival sequence.exportPNGandexportCanvascallflushImagesthemselves.mountNavigatedis async: the block decode is afetchof adata:URL.Tests
setImage, all checked to fail against the old renderer. Whole suite 2283 passed, 58 skipped.Review fixes, landed
rangewidget was ignored; overlays replaced a figure's own markers instead of merging by id; a ragged block read wrong rows when the index rank mismatched (it now throws).chipsbecameviews(the block switch above); a frame's display window defaults to the panel's own instead of a per-frame percentile pass; a size-changingsetImagecommits geometry and bytes in the same frame (first push 110 ms to 0.2 ms); the pixel sequence is global so two identical figures in one document no longer collide; the readers are exported under oneembednamespace instead of as generic top-level names.einsum. Whole suite after the fixes: 2301 passed, 58 skipped.