From 285574d9e7a8618b8b3dc7f91584e32970e1914f Mon Sep 17 00:00:00 2001 From: Max Inden Date: Thu, 18 Aug 2022 15:38:23 +0900 Subject: [PATCH 01/11] webrtc/: Add message framing to support half-close and reset of stream The WebRTC browser APIs do not support half-closing nor resets of streams. This commit defines a message framing schema to support this functionality on top of the browser APIs. --- webrtc/README.md | 76 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 60 insertions(+), 16 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 4b1a33370..b866298a4 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -227,7 +227,9 @@ After [Connection Establishment](#connection-establishment): fingerprints of _A_ and _B_ in their multihash byte representation, sorted in ascending order. -3. See [Multiplexing](#multiplexing). +3. On success of the authentication handshake, the used datachannel is + closed and the plain WebRTC connection is used with its multiplexing + capabilities via datachannels. See [Multiplexing](#multiplexing). Note: WebRTC supports different hash functions to hash the TLS certificate (see https://datatracker.ietf.org/doc/html/rfc8122#section-5). The hash function used @@ -260,21 +262,50 @@ be the same. On mismatch the final Noise handshake MUST fail. ## Multiplexing -After [Connection Security](#connection-security): - -1. On success of the authentication handshake _X_, the used datachannel is - closed and the plain WebRTC connection is used with its multiplexing - capabilities via datachannels. - -### Open Questions - -- Can we use WebRTC’s data channels in _Browser_ to multiplex a single - connection, or do we need to run an additional multiplexer (e.g. yamux) on top - of a WebRTC connection and WebRTC datachannel? In other words, does WebRTC - provide all functionality of a libp2p muxer like Yamux (e.g. flow control)? - - Yes, with WebRTC's datachannels running on top of SCTP, there is no need for - additional multiplexing. +Following [Connection Security](#connection-security). + +The WebRTC browser APIs do not support half-closing nor resets of streams. +[`RTCDataChannel.close()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel/close) +flushes the remaining messages and closes the local write and read side. After +calling `RTCDataChannel.close()` one can no longer read from the channel. This +lack of functionality is problematic, given that libp2p protocols running on top +of transport protocols, like WebRTC, expect to be able to half-close or reset a +stream. See [Connection Establishment in +libp2p](https://github.com/libp2p/specs/blob/master/connections/README.md#definitions). + +To support half-closing and resets of streams, libp2p WebRTC uses message +framing. Messages on a `RTCDataChannel` are embedded into the Protobuf message +below and send on the `RTCDataChannel` prefixed with the message length in +bytes, encoded as an unsigned variable length integer as defined by the +[multiformats unsigned-varint spec][uvarint-spec]. + +``` proto +syntax = "proto2"; + +package webrtc.pb; + +message Message { + enum Flag { + // The local endpoint will no longer send messages. + CLOSE_WRITE = 0; + // The local endpoint will no longer read messages. + CLOSE_READ = 1; + // The local endpoint abruptly terminates the stream. The remote endpoint + // may discard any in-flight data. + RESET = 2; + } + + optional Flag flag=1; + + optional bytes message = 2; +} +``` + +Encoded messages including their length prefix MUST NOT exceed 16kiB to support +all major browsers. See ["Understanding message size +limits"](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Using_data_channels#understanding_message_size_limits). +Implementations MAY choose to send smaller messages, e.g. to reduce delays +sending _flagged_ messages. ## General Open Questions @@ -339,3 +370,16 @@ After [Connection Security](#connection-security): the signature (`signature_libp2p_a(fingerprint_a, fingerprint_b, connection_identifier)`) would protect against this attack. To the best of our knowledge the browser does not give us access to such identifier. + +- _Why use Protobuf for WebRTC message framing. Why not use our own, + potentially smaller encoding schema?_ + + The Protobuf framing adds an overhead of 5 bytes. The unsigned-varint prefix + adds another 2 bytes. On a large message the overhead is negligible (`(5 + bytes + 2 bytes) / (16384 bytes - 7 bytes) = 0.000427246`). On a small + message, e.g. a multistream-select message with ~40 bytes the overhead is high + (`(5 bytes + 2 bytes) / 40 bytes = 0.175`) but likely irrelevant. + + Using Protobuf allows us to evolve the protocol in a backwards compatibile way + going forward. Using Protobuf is consisten with the many other libp2p + protocols. These benefits outweigh the drawback of additional overhead. From 373bafe204f94b1bfcc338fbe670680b5e19d9db Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 22 Aug 2022 17:55:05 +0900 Subject: [PATCH 02/11] webrtc/: Reword protobuf comment from local endpoint to sender --- webrtc/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index b866298a4..c0c00166f 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -286,9 +286,9 @@ package webrtc.pb; message Message { enum Flag { - // The local endpoint will no longer send messages. + // The sender will no longer send messages. CLOSE_WRITE = 0; - // The local endpoint will no longer read messages. + // The sender will no longer read messages. CLOSE_READ = 1; // The local endpoint abruptly terminates the stream. The remote endpoint // may discard any in-flight data. From 93df7e397e7ba79962ed7bdfbf6d4da40985011c Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 5 Sep 2022 10:59:00 +0900 Subject: [PATCH 03/11] webrtc/: Adapt enum variants to QUIC RFC --- webrtc/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index c0c00166f..930400c14 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -287,9 +287,10 @@ package webrtc.pb; message Message { enum Flag { // The sender will no longer send messages. - CLOSE_WRITE = 0; - // The sender will no longer read messages. - CLOSE_READ = 1; + FIN = 0; + // The sender will no longer read messages. Incoming data is being + // discarded on receipt. + STOP_SENDING = 1; // The local endpoint abruptly terminates the stream. The remote endpoint // may discard any in-flight data. RESET = 2; From 027b539c3292d0d2bec2b04307a484d940d6b255 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 5 Sep 2022 10:59:32 +0900 Subject: [PATCH 04/11] webrtc/: Link QUIC RFC as reference --- webrtc/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/webrtc/README.md b/webrtc/README.md index 930400c14..848e89ba8 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -302,6 +302,9 @@ message Message { } ``` +The above is adapted from the [QUIC RFC]. When in doubt on the semantics of +these messages, consult the [QUIC RFC]. + Encoded messages including their length prefix MUST NOT exceed 16kiB to support all major browsers. See ["Understanding message size limits"](https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Using_data_channels#understanding_message_size_limits). @@ -384,3 +387,5 @@ sending _flagged_ messages. Using Protobuf allows us to evolve the protocol in a backwards compatibile way going forward. Using Protobuf is consisten with the many other libp2p protocols. These benefits outweigh the drawback of additional overhead. + +[QUIC RFC]: https://www.rfc-editor.org/rfc/rfc9000.html From 865f4f2dea8872c8de301a16b59000ac4540f18d Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 5 Sep 2022 11:07:43 +0900 Subject: [PATCH 05/11] webrtc/: Redefine RESET semantics Instead of resetting both ends of a stream, RESET only resets the sending side of a stream. This is in line with the QUIC stream reset mechanism. --- webrtc/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index 848e89ba8..5be597bdf 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -286,13 +286,13 @@ package webrtc.pb; message Message { enum Flag { - // The sender will no longer send messages. + // The sender will no longer send messages on the stream. FIN = 0; - // The sender will no longer read messages. Incoming data is being - // discarded on receipt. + // The sender will no longer read messages on the stream. Incoming data is + // being discarded on receipt. STOP_SENDING = 1; - // The local endpoint abruptly terminates the stream. The remote endpoint - // may discard any in-flight data. + // The sender abruptly terminates the sending part of the stream. The + // receiver can discard any data that it already received on that stream. RESET = 2; } From a60234cfb44fe060c8448d0de6e91958a57fedbc Mon Sep 17 00:00:00 2001 From: Max Inden Date: Mon, 5 Sep 2022 11:19:39 +0900 Subject: [PATCH 06/11] webrtc/: Rephrase missing mechanism in browser webrtc api --- webrtc/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/webrtc/README.md b/webrtc/README.md index 5be597bdf..e5d24d3e1 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -264,7 +264,8 @@ be the same. On mismatch the final Noise handshake MUST fail. Following [Connection Security](#connection-security). -The WebRTC browser APIs do not support half-closing nor resets of streams. +The WebRTC browser APIs do not support half-closing of streams nor resets of the +sending part of streams. [`RTCDataChannel.close()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel/close) flushes the remaining messages and closes the local write and read side. After calling `RTCDataChannel.close()` one can no longer read from the channel. This From 7f404912eae8cf5beda3c50f4fdcb92afbf365ac Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sat, 10 Sep 2022 11:13:30 +0200 Subject: [PATCH 07/11] Apply suggestions from code review Co-authored-by: Thomas Eizinger --- webrtc/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index e5d24d3e1..e83e22f92 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -276,7 +276,7 @@ libp2p](https://github.com/libp2p/specs/blob/master/connections/README.md#defini To support half-closing and resets of streams, libp2p WebRTC uses message framing. Messages on a `RTCDataChannel` are embedded into the Protobuf message -below and send on the `RTCDataChannel` prefixed with the message length in +below and sent on the `RTCDataChannel` prefixed with the message length in bytes, encoded as an unsigned variable length integer as defined by the [multiformats unsigned-varint spec][uvarint-spec]. @@ -386,7 +386,7 @@ sending _flagged_ messages. (`(5 bytes + 2 bytes) / 40 bytes = 0.175`) but likely irrelevant. Using Protobuf allows us to evolve the protocol in a backwards compatibile way - going forward. Using Protobuf is consisten with the many other libp2p + going forward. Using Protobuf is consistent with the many other libp2p protocols. These benefits outweigh the drawback of additional overhead. [QUIC RFC]: https://www.rfc-editor.org/rfc/rfc9000.html From 31ac65d6a78f621babd8d6c2845881b1128db4ad Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sat, 10 Sep 2022 18:25:22 +0900 Subject: [PATCH 08/11] webrtc/: Require RESET after STOP_SENDING --- webrtc/README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index e83e22f92..77bf0ad02 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -280,6 +280,9 @@ below and sent on the `RTCDataChannel` prefixed with the message length in bytes, encoded as an unsigned variable length integer as defined by the [multiformats unsigned-varint spec][uvarint-spec]. +It is an adaptation from the [QUIC RFC]. When in doubt on the semantics of +these messages, consult the [QUIC RFC]. + ``` proto syntax = "proto2"; @@ -303,8 +306,9 @@ message Message { } ``` -The above is adapted from the [QUIC RFC]. When in doubt on the semantics of -these messages, consult the [QUIC RFC]. +Note that "a STOP_SENDING frame requests that the receiving endpoint send a +RESET_STREAM frame.". See [QUIC RFC - 3.5 Solicited State +Transitions](https://www.rfc-editor.org/rfc/rfc9000.html#section-3.5). Encoded messages including their length prefix MUST NOT exceed 16kiB to support all major browsers. See ["Understanding message size From 85364f44d5b6d0461892ebcb2e5f2d13afe2df28 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sat, 10 Sep 2022 18:27:37 +0900 Subject: [PATCH 09/11] webrtc/: Rename RESET to RESET_STREAM For the sake of consistency with the QUIC RFC, rename RESET to RESET_STREAM. --- webrtc/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/webrtc/README.md b/webrtc/README.md index 77bf0ad02..4281ebfc8 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -297,7 +297,7 @@ message Message { STOP_SENDING = 1; // The sender abruptly terminates the sending part of the stream. The // receiver can discard any data that it already received on that stream. - RESET = 2; + RESET_STREAM = 2; } optional Flag flag=1; From 7a8ebc03c1b514922adfbec6e6ac756c0c892599 Mon Sep 17 00:00:00 2001 From: Max Inden Date: Sun, 11 Sep 2022 18:00:31 +0900 Subject: [PATCH 10/11] webrtc/: Use message framing for Noise handshake already Instead of executing the additional Noise handshake on a plain `RTCDataChannel`, frame the Noise messages with the proposed message framing mechanism already. Benefits: - The additional Noise handshake can already make use of the proper closing mechanism. Note that `RTCDataChannel.close` may drop any in-flight data. See https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel/close. - Simplifies implementations as there is only one way to write on an `RTCDataChannel`. - Allows us to role out new versions of the Noise handshake in the future by adding a version to the message framing Protobuf. --- webrtc/README.md | 127 ++++++++++++++++++++++++----------------------- 1 file changed, 65 insertions(+), 62 deletions(-) diff --git a/webrtc/README.md b/webrtc/README.md index c592ef766..2b18bcbd9 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -20,10 +20,10 @@ Interest Group: [@marten-seemann] - [Open Questions](#open-questions) - [Browser to Browser](#browser-to-browser) - [Open Questions](#open-questions-1) - - [Connection Security](#connection-security) - - [Open Questions](#open-questions-2) - [Multiplexing](#multiplexing) - [Ordering](#ordering) + - [Connection Security](#connection-security) + - [Open Questions](#open-questions-2) - [General Open Questions](#general-open-questions) - [Previous, ongoing and related work](#previous-ongoing-and-related-work) - [FAQ](#faq) @@ -116,9 +116,11 @@ fingerprint](https://www.w3.org/TR/webrtc/#dom-rtccertificate-getfingerprints). fingerprint through the WebRTC DTLS handshake. At this point the DTLS handshake provides confidentiality and integrity but not authenticity. -7. See [Connection Security](#connection-security). +7. Messages on an `RTCDataChannel` are framed using the message framing + mechanism described in [Multiplexing](#multiplexing). -8. See [Multiplexing](#multiplexing). +7. The remote is authenticated via an additional Noise handshake. See + [Connection Security](#connection-security). #### Open Questions @@ -210,66 +212,8 @@ server node _R_. could they exchange their multiaddr and construct the remote's SDP packet based on it? -## Connection Security - -While WebRTC offers confidentiality and integrity via TLS, one still needs to -authenticate the remote peer by its libp2p identity. - -After [Connection Establishment](#connection-establishment): - -1. _A_ opens a WebRTC datachannel. - -2. _A_ starts a Noise `XX` handshake using _A_'s and _B_'s libp2p identity. See - [noise-libp2p](https://github.com/libp2p/specs/tree/master/noise). - - Instead of exchanging the TLS certificate fingerprints on the established - Noise channel once the Noise handshake succeeded, _A_ and _B_ use the [Noise - Prologue](https://noiseprotocol.org/noise.html#prologue) mechanism, thus - saving one round trip. - - More specifically _A_ and _B_ set the Noise _Prologue_ to - `libp2p-webrtc-noise:` before starting the actual Noise - handshake. `` is the concatenation of the of the two TLS - fingerprints of _A_ and _B_ in their multihash byte representation, sorted in - ascending order. - -3. On success of the authentication handshake, the used datachannel is - closed and the plain WebRTC connection is used with its multiplexing - capabilities via datachannels. See [Multiplexing](#multiplexing). - -Note: WebRTC supports different hash functions to hash the TLS certificate (see -https://datatracker.ietf.org/doc/html/rfc8122#section-5). The hash function used -in WebRTC and the hash function used in the multiaddr `/certhash` component MUST -be the same. On mismatch the final Noise handshake MUST fail. - -### Open Questions - -- Can a _Browser_ access the fingerprint of its TLS certificate? - - Chrome allows you to access the fingerprint of any locally-created certificate - directly via `RTCCertificate#getFingerprints`. Firefox does not allow you to - do so. Browser compatibility can be found - [here](https://developer.mozilla.org/en-US/docs/Web/API/RTCCertificate). In - practice, this is not an issue since the fingerprint is embedded in the local - SDP string. - -- Is the above proposed additional handshake secure? See also alternative - proposed Handshake for - [WebTransport](https://github.com/libp2p/specs/pull/404). - -- Would it be more efficient for _B_ to initiate the Noise handshake? In other - words, who is able to write on an established WebRTC connection first? _A_ or - _B_? - -- On the server side, can one derive the TLS certificate in a deterministic way - based on a node's libp2p private key? Benefit would be that a node only needs - to persist the libp2p private key and not the TLS key material while still - maintaining a fixed TLS certificate fingerprint. - ## Multiplexing -Following [Connection Security](#connection-security). - The WebRTC browser APIs do not support half-closing of streams nor resets of the sending part of streams. [`RTCDataChannel.close()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel/close) @@ -329,6 +273,65 @@ overriding the default value of `ordered` `true` to `false` when creating a new data channel via [`RTCPeerConnection.createDataChannel`](https://www.w3.org/TR/webrtc/#dom-peerconnection-createdatachannel). +## Connection Security + +Note that the below uses the message framing described in +[multiplexing](#multiplexing). + +While WebRTC offers confidentiality and integrity via TLS, one still needs to +authenticate the remote peer by its libp2p identity. + +After [Connection Establishment](#connection-establishment): + +1. _A_ opens a WebRTC datachannel. + +2. _A_ starts a Noise `XX` handshake using _A_'s and _B_'s libp2p identity. See + [noise-libp2p](https://github.com/libp2p/specs/tree/master/noise). + + Instead of exchanging the TLS certificate fingerprints on the established + Noise channel once the Noise handshake succeeded, _A_ and _B_ use the [Noise + Prologue](https://noiseprotocol.org/noise.html#prologue) mechanism, thus + saving one round trip. + + More specifically _A_ and _B_ set the Noise _Prologue_ to + `libp2p-webrtc-noise:` before starting the actual Noise + handshake. `` is the concatenation of the of the two TLS + fingerprints of _A_ and _B_ in their multihash byte representation, sorted in + ascending order. + +3. On success of the authentication handshake, the used datachannel is + closed and the plain WebRTC connection is used with its multiplexing + capabilities via datachannels. See [Multiplexing](#multiplexing). + +Note: WebRTC supports different hash functions to hash the TLS certificate (see +https://datatracker.ietf.org/doc/html/rfc8122#section-5). The hash function used +in WebRTC and the hash function used in the multiaddr `/certhash` component MUST +be the same. On mismatch the final Noise handshake MUST fail. + +### Open Questions + +- Can a _Browser_ access the fingerprint of its TLS certificate? + + Chrome allows you to access the fingerprint of any locally-created certificate + directly via `RTCCertificate#getFingerprints`. Firefox does not allow you to + do so. Browser compatibility can be found + [here](https://developer.mozilla.org/en-US/docs/Web/API/RTCCertificate). In + practice, this is not an issue since the fingerprint is embedded in the local + SDP string. + +- Is the above proposed additional handshake secure? See also alternative + proposed Handshake for + [WebTransport](https://github.com/libp2p/specs/pull/404). + +- Would it be more efficient for _B_ to initiate the Noise handshake? In other + words, who is able to write on an established WebRTC connection first? _A_ or + _B_? + +- On the server side, can one derive the TLS certificate in a deterministic way + based on a node's libp2p private key? Benefit would be that a node only needs + to persist the libp2p private key and not the TLS key material while still + maintaining a fixed TLS certificate fingerprint. + ## General Open Questions - Should libp2p's WebRTC stack limit itself to using UDP only, or support WebRTC From f0acbb58302453e16df14cef84ccad23f18a307e Mon Sep 17 00:00:00 2001 From: Max Inden Date: Tue, 13 Sep 2022 05:17:34 +0200 Subject: [PATCH 11/11] Update webrtc/README.md Co-authored-by: Thomas Eizinger --- webrtc/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/webrtc/README.md b/webrtc/README.md index 2b18bcbd9..f508ec589 100644 --- a/webrtc/README.md +++ b/webrtc/README.md @@ -119,7 +119,7 @@ fingerprint](https://www.w3.org/TR/webrtc/#dom-rtccertificate-getfingerprints). 7. Messages on an `RTCDataChannel` are framed using the message framing mechanism described in [Multiplexing](#multiplexing). -7. The remote is authenticated via an additional Noise handshake. See +8. The remote is authenticated via an additional Noise handshake. See [Connection Security](#connection-security). #### Open Questions