bug: resource usage

[erhant]

Problem

The process may sometimes reach ulimit and therefore give errors during API calls such as Os { code: 24, kind: Uncategorized, message: \"No file descriptors available\" })) } during a DNS lookup.

How to Reproduce

Not yet known.

Expected Behaviour

Should not give error.

Version

• 0.2.18

[erhant]

hyperium/hyper#1422 related?

[erhant]

https://discuss.libp2p.io/t/rust-reading-from-socket-slower-with-every-event/460/2 also related maybe

[erhant]

by using lsof -c <process-name-here> on the compute node, we have seen that as the peer count grows file descriptors are used more and more, eventually hitting the default limit of 1024.

on a separate note, the peer count is almost 1-1 correlated with the established outgoing connections count, so as a quick solution we have added a limiter on this count, effectively setting a cap on the number of all peers of a node.

[erhant]

setting the maximum established outgoing connections allows to set a cap on this resource usage

[erhant]

with libp2p/rust-libp2p#5595 this may be fixed

[erhant]

Launcher now has a much better method of setting resource limits, see https://github.com/firstbatchxyz/dkn-compute-launcher/blob/master/src/utils/fdlimit.rs

We no longer have this issue.