Skip to content

Add a spatial index#83

Open
boydjohnson wants to merge 16 commits into
boxdot:masterfrom
boydjohnson:feature/spatial-index
Open

Add a spatial index#83
boydjohnson wants to merge 16 commits into
boxdot:masterfrom
boydjohnson:feature/spatial-index

Conversation

@boydjohnson

Copy link
Copy Markdown

The key insight was that the nodes, ways, and relations could be sorted in the curve's order and then found via binary search at query time. The schema is modified for relations to include the minimum bounding box, because that is often difficult to find at query time.

Entity Curve Resolution Domain Indexed on Notes
Nodes ZCurve2D (Z-order / Morton) 1024 (curve default) lon/lat, full WGS84 (-180..180, -90..90) point (lon, lat) Points have no extent, so a plain Z-curve suffices. Default WGS84 bounds.
Ways XZ2SFC (extended Z-order) SPATIAL_RESOLUTION = 18 XZ2SFC::wgs84(18) minimum bounding box (min_x, min_y, max_x, max_y) XZ2 handles entities with extent (boxes). Ordered by bbox index.
Relations XZ2SFC (extended Z-order) SPATIAL_RESOLUTION = 18 XZ2SFC::wgs84(18) minimum bounding box (mbb [i32; 4]) Same curve as ways (way_curve()). Bbox stored in the archive.

osmflatc now uses RocksDB to do the ordering based on the curve's index. There are osmflatc arguments for configuring RocksDB.

--scratch-dir <SCRATCH_DIR>
Scratch RocksDB directory; use a fast SSD with ample space [default: output's parent]
--block-cache-mb <BLOCK_CACHE_MB>
Scratch RocksDB block cache (MiB); raise (e.g. 8192) for planet-scale [default: 512]
--write-buffer-mb <WRITE_BUFFER_MB>
Scratch RocksDB write buffer (MiB); raise for higher write throughput [default: 128]
--max-open-files <MAX_OPEN_FILES>
Max open SST files RocksDB may cache (-1 = unlimited). Must stay below the process open-file limit (raise it with ulimit -n); lower this if you hit "Too many open files" [default: -1]

Note: Some of the code included in this PR was written by Claude Code.

@boydjohnson

Copy link
Copy Markdown
Author

This PR was talked about in #81

@boydjohnson boydjohnson marked this pull request as draft June 29, 2026 21:23
@boydjohnson

Copy link
Copy Markdown
Author

I realized I messed up the --ids index. Before it was implicitly id ordered, as well as all the nodes, ways, and relations. So before Entity => Id was O(1), and then binary search for Id => Entity.

With this PR, Entity => Id is still O(1) but there is no binary search for Id => Entity.

I am going to add an index that will do the --reverse-ids.

@boydjohnson boydjohnson marked this pull request as ready for review June 29, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant