QUIC "Message too long"

[nazar-pc]

Summary

With switch to QUIC I sometimes getting messages like this:

2023-10-14T07:49:21.668469Z sendmsg error: Os { code: 90, kind: Uncategorized, message: "Message too long" }, Transmit: { destination: 65.108.232.15:30533, src_ip: None, enc: Some(Ect0), len: 1420, segment_size: None }

Expected behavior

Should not exceed MTU

Actual behavior

Fails, likely due to exceeding MTU

Relevant log output

No response

Possible Solution

This likely happens because I have MTU set to 1420 (Wireguard is the reason) rather than more common 1500. If there is MTU expectation hardcoded somewhere it probably needs to change to MTU discovery instead. If MTU discovery is already present, it should not print warnings/errors.

Changing MTU back to 1500 resolves the issue.

Version

libp2p 0.52.3
libp2p-quic 0.9.2

Would you like to work on fixing this bug ?

Maybe

[thomaseizinger]

Yeah, the default MTU of QUIC is 1500 - 28: https://docs.rs/quinn-proto/latest/src/quinn_proto/config.rs.html#517

I guess we could add a configuration flag to change this!

[nazar-pc]

I don't think we'll be changing it in our implementation either, it ideally needs to adjust to whatever actual MTU is when connection is established or message size when this error is hit.

[thomaseizinger]

I don't think we'll be changing it in our implementation either, it ideally needs to adjust to whatever actual MTU is when connection is established or message size when this error is hit.

Is that possible? Some quick research suggests that even discovering the MTU of the local interface is not all that useful and instead you'd need to do MTU path discovery.

Seems to be much easier to start with configuring it first if you know that you'll be running over wireguard for example.

[mxinden]

@nazar-pc can you test with MtuDiscoveryConfig::default()?

https://docs.rs/quinn/latest/quinn/struct.TransportConfig.html#method.mtu_discovery_config

Additional context, in case folks want to dive deeper: https://www.rfc-editor.org/rfc/rfc9000.html#name-datagram-size

[nazar-pc]

@nazar-pc can you test with MtuDiscoveryConfig::default()?

Isn't MtuDiscoveryConfig::default() what is used by default already though?

[thomaseizinger]

@nazar-pc can you test with MtuDiscoveryConfig::default()?

Isn't MtuDiscoveryConfig::default() what is used by default already though?

From what I know, yes.

Where exactly is that log emitted? Perhaps it is an issue within quinn?

[nazar-pc]

Where exactly is that log emitted? Perhaps it is an issue within quinn?

It comes from quinn_udp according to this report for example.

[thomaseizinger]

Where exactly is that log emitted? Perhaps it is an issue within quinn?

It comes from quinn_udp according to this report for example.

Could you investigate upstream? I am not really sure what we can do in libp2p if we are already using path MTU discovery.