Implement private networks

[rklaehn]

Implements private networks ( #476 )

This is meant to be interoperable with go-ipfs private swarms. For reference: the specification from PL is at https://github.com/libp2p/specs/blob/master/pnet/Private-Networks-PSK-V1.md , the implementation in go-ipfs is at https://github.com/libp2p/go-libp2p-pnet/ .

Tested by checking that two rust-ipfs nodes (pnet example) can chat with each other, and in addition checking that the pubsub traffic is visible in go-ipfs.

[rklaehn]

I think the basic encryption is now reasonably correct. Here is the spec for reference: https://github.com/libp2p/specs/blob/master/pnet/Private-Networks-PSK-V1.md

However, I tried getting the chat example to work with go-ipfs with a private swarm key, but could not get it to work.

This might be related to the UpgradeInfo. I used a made up protocol name because I had to put something there. b"/pnet/1.0.0". But I suspect that that is causing trouble. I tried implementing an UpgradeInfo that returns an empty iterator, but with that I could not get it to work at all...

Looking at the help for the method:

    /// Returns the list of protocols that are supported. Used during the negotiation process.
    fn protocol_info(&self) -> Self::InfoIter;

It seems that what happens is that my made up protocol name will now be part of the negotiation process. So obviously this won't work since the go-ipfs side does not know about my made up protocol name. Is there a way around this?

[tomaka]

Oh I see, I didn't think about that.
I guess that you will have to apply the handshake manually over the connections that the Transport produces.
You can do that with transport.and_then(|socket, _| handshake(socket)).

[rklaehn]

Not exactly sure how to wire this up. I naively tried this:

pub fn build_tcp_ws_pnet_secio_mplex_yamux(keypair: identity::Keypair, psk: PSK)
    -> io::Result<impl Transport<Output = (PeerId, impl core::muxing::StreamMuxer<OutboundSubstream = impl Send, Substream = impl Send, Error = impl Into<io::Error>> + Send + Sync), Error = impl error::Error + Send, Listener = impl Send, Dial = impl Send, ListenerUpgrade = impl Send> + Clone>
{
    Ok(CommonTransport::new()?
        .and_then(|socket, endpoint| PnetConfig::new(psk).handshake(socket))
        // .and_then(move |io, endpoint| {
        //     libp2p_core::upgrade::apply(
        //         io,
        //         PnetConfig::new(psk),
        //         endpoint,
        //         libp2p_core::transport::upgrade::Version::V1,
        //     )
        // })
        .upgrade(core::upgrade::Version::V1)
        .authenticate(secio::SecioConfig::new(keypair))
        .multiplex(core::upgrade::SelectUpgrade::new(yamux::Config::default(), mplex::MplexConfig::new()))
        .map(|(peer, muxer), _| (peer, core::muxing::StreamMuxerBox::new(muxer)))
        .timeout(Duration::from_secs(20)))
}

...but then the result of the and_then is not a transport anymore so the .upgrade(core::upgrade::Version::V1) does not work anymore. I guess I have to call it a day...

[tomaka]

...but then the result of the and_then is not a transport anymore

I think that's fixed by #1384
Your async function probably produces a non-Unpin future that aren't supported now (but will be after #1384)

[rklaehn]

Right. Got it to work now. Making handshake return a BoxFuture fixes it!

    pub fn pin_handshake<TSocket>(self, socket: TSocket) -> BoxFuture<'static, Result<PnetOutput<TSocket>, PnetError>>
        where
            TSocket: AsyncRead + AsyncWrite + Send + Unpin + 'static,
    {
        Box::pin(self.handshake(socket))
    }

Got the thing to talk to go-ipfs now. Thanks for helping me sort this out!

[rklaehn]

I will remove or replace the pnet example before merging this. Now just want to keep it in for testing.

[tomaka]

To me, multiple calls to poll_write should continue adding data at the end of the buffer, and only call poll_flush_buf if there is no space left.
However I think the current approach is fine as well.

Looks mostly good for me, except that pnet/src/lib.rs needs documentation.

[tomaka]

LGTM. Please remove the example and let's merge!

[tomaka]

Wait, I didn't realize that you had changed the logic of the buffer writing.

Outdated

[rklaehn]

Happy to revert it if you want, but I think the new code is better.

[tomaka]

The buffering logic still looks hard to read to me, hence these debug_assert!s, after which we should be good for merging.

[rklaehn]

@tomaka I have taken all your suggestions and merged them into one commit, and rebased on current master.