V1.1 early draft preview#151
Conversation
|
|
||
| \*********************************************************************************************************************** | ||
| \* Subject-ID mapping function. The ring size is the total number of distinct subject-IDs. | ||
| \* TODO: Switch to quadratic probing: https://github.com/OpenCyphal-Garage/cy/issues/12 |
There was a problem hiding this comment.
It's a stale comment that should be removed. The model uses linear probing because it is simpler to reason about in the formal verification context. The actual implementation uses quadratic probing and this difference is believed to be insignificant for the purposes of the consensus protocol, as noted in the adjacent appendix Convergence proof for the allocation CRDT under Instantiating A3/A4 for common probing laws.
| \item ``Consistent Overhead Byte Stuffing'', Stuart Cheshire and Mary Baker. | ||
| \item Extracted the DSDL specification into a separate document. | ||
| \item Introduced named topics and the session layer, along with the distributed consensus algorithm. | ||
| \item Stabilized the experimental Cyphal/UDP transport. |
There was a problem hiding this comment.
We've broken it. We need to be clear that this is a non-backwards-compatible change but one we consider finalised. As discussed in the devcall; it's important we produce an appendix that is about UDP v1.0 -> v1.1 upgrade path and interoperability. We also need to produce a rationale for the breaking change.
| @@ -1,232 +0,0 @@ | |||
| \section{Cyphal/serial (experimental)}\label{sec:transport_serial} | |||
There was a problem hiding this comment.
Why not carry this forward while remaining experimental? If we did create a full version of this in v1.x we wouldn't want a gap where the specification had a serial protocol, dropped it, then introduced another one. It would be better to have had one, experimental protocol we finally got around to promoting.
There was a problem hiding this comment.
I removed it because it carries the same unwanted features we had in the old UDP transport, and at this point I chose to not define a new serial transport, chiefly because I don't have an immediate use case for it and I don't know anyone who does. I don't think it has seen any significant usage in the field otherwise we would have probably heard about it on the forum or elsewhere.
| For example, the pattern \texttt{sensors/*/data} matches \texttt{sensors/imu/data} | ||
| and \texttt{sensors/gps/data} but not \texttt{sensors/imu/raw/data}. | ||
|
|
||
| \item[\texttt{>} (ASCII $62$)] Matches zero or more trailing segments. |
There was a problem hiding this comment.
Claude flagged this and I agree we should clarify it so anyone using LLMs to generate parsers doesn't get this wrong:
Pattern-token boundary semantics for > are under-specified.
§2.3.2 says > matches "zero or more trailing segments" and gives the example that sensors/> matches sensors. After normalization (which "trims leading and trailing separators"), sensors/> does not get its / trimmed because > is not a separator, but matching a name with zero trailing segments against a pattern containing /> is only correct if the slash is treated as elidable when zero segments follow. The spec should explicitly state how the slash before > interacts with zero matches (currently inferred only from the example).
|
|
||
| Every session-layer message begins with a fixed-size header of 24 bytes, | ||
| followed by a type-specific payload. Multi-byte integer fields are encoded in | ||
| little-endian byte order and are positioned to favour natural alignment on |
There was a problem hiding this comment.
This is an LLM optimization as Claude panicked when it saw a uint48 that was not positioned on a natural alignment:
The spec could spell out this rationale once: "8-byte fields are placed on 8-byte boundaries; smaller integer fields are packed without further alignment." That would close off readers like me who try to extrapolate "favour" into something stronger than it is.
There was a problem hiding this comment.
Wording issue. What I actually meant when I wrote this in the source document (that Claude later transformed into this spec) is that the layout is easy to represent in a C structure without it being packed. I think it's best to just say it directly.
The idea is that sequences like uint48 followed by uint16 are represented as a single uint64_t. Now when I write this I realize that this is, indeed, packing, but it is manual and predictable, unlike __attribute__((packed)) et al.
| \input{introduction/introduction.tex} | ||
| \input{basic/basic.tex} | ||
| \input{dsdl/dsdl.tex} | ||
| \input{session/session.tex} |
There was a problem hiding this comment.
Shouldn't transport come first? (bottom up?)
There was a problem hiding this comment.
I'm fine either way but I think the session layer is the main character of this document so there might be some value in keeping it up-front?
| \begin{minted}{python} | ||
| uint8 type # 2 for msg_ack, 3 for msg_nack. | ||
| void24 | ||
| uint32 incompatibility |
There was a problem hiding this comment.
Why is incompatibility uint32 here and uint8 in the session_header_msg?
There was a problem hiding this comment.
Eh it just depends on the number of unused bits. More bits --> wider incompatibility field. No other reasoning.
|
|
||
| \subsection{Pinned topics} | ||
|
|
||
| Pinned topics are ordinary CRDT records with their eviction counter pre-set to |
There was a problem hiding this comment.
This seems to make V1.0 interop impossible, right?
There was a problem hiding this comment.
I should expand a bit on this topic. They are ordinary CDRT records, but the consensus protocol is not required for a consensus to be reached, because it can be enforced by configuring pinning on all nodes manually. I will fix this when I'm back to the spec.
|
Check v1.0 interop description |
2 participants
I wrote this today with heavy assistance from Opus. The content was generally generated based on the formal models and the reference implementation, both of which can be found in https://github.com/OpenCyphal-Garage/cy. That is the source of truth, while the specification is a mere compilation.
Content-wise this is nearly complete, but there are many stylistic issues that I have intentionally left unresolved for now. Please focus on the idea first of all and ignore the style.
The essential changes are:
The DSDL and application layer chapters are completely removed. This necessitated a few collateral changes throughout. I have created a new repository for the upcoming DSDL spec extraction but I am not planning to work on that anytime soon yet; help is welcome: https://github.com/OpenCyphal/dsdl_specification
The transport layer chapter was reworked to remove most of the transport-agnostic concepts, because they are not transport-agnostic anymore but rather CAN-specific. The CAN chapter therefore may seem heavily edited but it is actually not; the only substantive changes are the addition of the new 16-bit subject-ID CAN ID layout and notes on backward compatibility.
The Cyphal/UDP spec has been updated to match the new proposed UDP transport design. The old UDP transport is for now removed, but as we discussed at the last call we may elect to keep it under a different name. I prefer to remove the old one completely; if we keep it, someone must undertake to maintain it.
A completely new Session layer chapter is added.
Many auxiliary materials and design rationale are provided in the appendices, which are mostly derived from the formal models in the Cy repo.
I do not recommend looking at the diff. Download the built PDF from the CI and read that instead. Treat it as a new document and ignore the delta from v1.0 for now.