问题描述
当前 VOM 在 AX Tree、DOMSnapshot 和 iframe 数据尚未完成统一建模前，就开始过滤、降级和拼接节点，导致部分语义信息过早丢失。例如，没有 backendDOMNodeId 的 AX 容器会被忽略，generic 中间节点被压缩后可能使工具栏、分组等结构变得扁平；已有交互角色但缺少 AX name 的按钮也无法统一利用 aria-label、title 或可见子文本恢复名称。跨 iframe 合并后，还可能出现父页面 context 传播到 iframe 内控件的问题。
可能的解决方案
引入统一、无损的 VOM 语义图：分别采集各 frame 的 AX Tree、DOMSnapshot、布局信息和 CDP session，以 frameId + sourceNodeId 作为节点标识，在同一张图中保留 AX、DOM 和 Frame 三类父子关系。没有 backend id 的 AX 节点仍参与语义结构，但不生成交互 ref；所有 frame 完成合并后，再统一执行角色解析、名称解析、结构归一化和渲染。
名称解析应对所有节点采用一致的优先级，例如 AX computed name、aria-labelledby、aria-label、关联 label、title、可见子文本和稳定图标提示。结构压缩则只应移除没有名称、状态、ARIA 关系或分组意义的透明节点，并将 FrameBoundary 作为明确的结构、坐标、session 和 context 边界，避免 iframe 内外语义污染。
效率
该重构不应增加逐节点 CDP 请求，整体处理应保持线性复杂度。需要补充普通页面、多 iframe、嵌套 iframe、大节点量页面的正确性与性能测试，确保语义结构改善的同时，不显著增加观察延迟和最终 VOM token 数。
Problem Description
The current VOM begins filtering, degrading, and merging nodes before the AX Tree, DOMSnapshot, and iframe data have been modeled consistently. This causes semantic information to be discarded too early. For example, AX containers without a backendDOMNodeId are ignored, collapsing intermediate generic nodes can flatten meaningful structures such as toolbars and groups, and interactive elements without an AX name cannot consistently recover names from aria-label, title, or visible descendant text. After iframe merging, context from the parent page may also incorrectly propagate to controls inside the iframe.
Proposed Solution
Introduce a unified, lossless VOM semantic graph. Capture the AX Tree, DOMSnapshot, layout information, and CDP session for each frame, and identify nodes using frameId + sourceNodeId. Preserve AX, DOM, and frame parent relationships within the same graph. AX nodes without a backend node ID should still participate in the semantic structure but should not generate interactive refs. Role resolution, name resolution, structural normalization, and rendering should run only after all frames have been merged.
Name resolution should use a consistent priority for every node, such as the AX computed name, aria-labelledby, aria-label, associated labels, title, visible descendant text, and stable icon hints. Structural compression should remove only transparent nodes that have no name, state, ARIA relationship, or grouping purpose. A FrameBoundary should act as an explicit structure, coordinate, session, and context boundary to prevent semantic context from leaking between the parent page and iframe contents.
Performance
This refactor should not introduce per-node CDP requests, and the overall processing should remain linear in the number of AX and DOM nodes. Correctness and performance tests should cover regular pages, multiple iframes, nested iframes, and pages with large node counts, ensuring that semantic quality improves without materially increasing observation latency or final VOM token usage.
问题描述
当前 VOM 在 AX Tree、DOMSnapshot 和 iframe 数据尚未完成统一建模前，就开始过滤、降级和拼接节点，导致部分语义信息过早丢失。例如，没有 backendDOMNodeId 的 AX 容器会被忽略，generic 中间节点被压缩后可能使工具栏、分组等结构变得扁平；已有交互角色但缺少 AX name 的按钮也无法统一利用 aria-label、title 或可见子文本恢复名称。跨 iframe 合并后，还可能出现父页面 context 传播到 iframe 内控件的问题。
可能的解决方案
引入统一、无损的 VOM 语义图：分别采集各 frame 的 AX Tree、DOMSnapshot、布局信息和 CDP session，以 frameId + sourceNodeId 作为节点标识，在同一张图中保留 AX、DOM 和 Frame 三类父子关系。没有 backend id 的 AX 节点仍参与语义结构，但不生成交互 ref；所有 frame 完成合并后，再统一执行角色解析、名称解析、结构归一化和渲染。
名称解析应对所有节点采用一致的优先级，例如 AX computed name、aria-labelledby、aria-label、关联 label、title、可见子文本和稳定图标提示。结构压缩则只应移除没有名称、状态、ARIA 关系或分组意义的透明节点，并将 FrameBoundary 作为明确的结构、坐标、session 和 context 边界，避免 iframe 内外语义污染。
效率
该重构不应增加逐节点 CDP 请求，整体处理应保持线性复杂度。需要补充普通页面、多 iframe、嵌套 iframe、大节点量页面的正确性与性能测试，确保语义结构改善的同时，不显著增加观察延迟和最终 VOM token 数。
Problem Description
The current VOM begins filtering, degrading, and merging nodes before the AX Tree, DOMSnapshot, and iframe data have been modeled consistently. This causes semantic information to be discarded too early. For example, AX containers without a backendDOMNodeId are ignored, collapsing intermediate generic nodes can flatten meaningful structures such as toolbars and groups, and interactive elements without an AX name cannot consistently recover names from aria-label, title, or visible descendant text. After iframe merging, context from the parent page may also incorrectly propagate to controls inside the iframe.
Proposed Solution
Introduce a unified, lossless VOM semantic graph. Capture the AX Tree, DOMSnapshot, layout information, and CDP session for each frame, and identify nodes using frameId + sourceNodeId. Preserve AX, DOM, and frame parent relationships within the same graph. AX nodes without a backend node ID should still participate in the semantic structure but should not generate interactive refs. Role resolution, name resolution, structural normalization, and rendering should run only after all frames have been merged.
Name resolution should use a consistent priority for every node, such as the AX computed name, aria-labelledby, aria-label, associated labels, title, visible descendant text, and stable icon hints. Structural compression should remove only transparent nodes that have no name, state, ARIA relationship, or grouping purpose. A FrameBoundary should act as an explicit structure, coordinate, session, and context boundary to prevent semantic context from leaking between the parent page and iframe contents.
Performance
This refactor should not introduce per-node CDP requests, and the overall processing should remain linear in the number of AX and DOM nodes. Correctness and performance tests should cover regular pages, multiple iframes, nested iframes, and pages with large node counts, ensuring that semantic quality improves without materially increasing observation latency or final VOM token usage.