Why does cargo download submodules when using a git dependency?

[ia0]

Problem

Why does cargo download submodules when using a git dependency? I couldn't find a way to disable it. This is an issue because there's no need to download so much stuff.

Other related issues I found seem to assume this is normal:

• Failed to update git submodule of dependency  #9622
• git submodules are not cached #7987

Steps

No response

Possible Solution(s)

No response

Notes

No response

Version

[epage]

I'm assuming its because Cargo can't know if the content of the submodule is relevant.

For instance, toml-test-data uses a submodule to pull in third-party data files that get built into the crate. Without loading the submodule, a git dependency on toml-test-data would fail to build.

[ia0]

That's what I assumed. But then why isn't there an option to say that submodules are irrelevant?

[epage]

It appears no one has requested something like that yet (not seeing any rejected issues or any Internals posts). Unsure about the team in the past but I find it best for us to evaluate features when there is a real user, rather than speculatively developing them. Even then, we need to weigh out the benefits vs the usability and maintenance costs.

It would be helpful to know more about the problem. What kind of repos are you using that you don't need the submodule? How many are there? What is the impact?

[ia0]

It's only one repository, that's the first time I notice this because it's quite rare to depend on a git repo that has big useless submodules.

This is the dependency that has big useless submodules: https://github.com/google/wasefire/blob/bf5a16ad8835cb6369bdf53c26bc57552aea83cc/examples/rust/opensk/Cargo.toml#L13

In particular the following submodule: https://github.com/tock/tock.

The reason it is useless is because the git repository of that dependency contains both a library (which I depend on) and a binary (which I don't depend on). The submodules are needed for the binary only.

EDIT: Regarding impact, I did not measure but it's maybe 3 to 4 seconds when building. Ignoring the storage aspect of cloning big useless submodules.

[epage]

EDIT: Regarding impact, I did not measure but it's maybe 3 to 4 seconds when building. Ignoring the storage aspect of cloning big useless submodules.

We had a perf hit from submodules that was fixed by #14605 which was released in 1.84.

I assume you are on a version newer than 1.84, so it would be good to dig more into why there is a 3-4 second hit per build. If you want, you can use CARGO_LOG_PROFILE to get some insights, see https://doc.crates.io/contrib/tests/profiling.html

[weihanglo]

The workaround is letting the git repo to set submodule.<name>.update = none if that one is meant to be pretty optional. Cargo respects that since 1.63 #10717 .

BTW, this is probably a duplicate of #4247.

[epage]

Ah missed that, closing in favor of #4247. Let us know if there is a reason to keep this open.

[ia0]

How did I miss that issue... Thanks!

[inferiorhumanorgans]

It appears no one has requested something like that yet (not seeing any rejected issues or any Internals posts). Unsure about the team in the past but I find it best for us to evaluate features when there is a real user, rather than speculatively developing them. Even then, we need to weigh out the benefits vs the usability and maintenance costs.

It would be helpful to know more about the problem. What kind of repos are you using that you don't need the submodule? How many are there? What is the impact?

@epage See #4247. This has been outstanding for almost a decade.