cargo package --list --manifest-path in a workspace prints only Cargo.lock on macos

Problem

While cargo cargo package --list itself works on all platforms and cargo package --list --manifest-path works on linux and windows, cargo package --list --manifest-path on mac in a workspace prints only Cargo.lock

I've made a reproducer where you can see the results for the three platforms in github actions: https://github.com/PyO3/maturin/runs/5535598729?check_suite_focus=true

Steps

cd /tmp
git clone https://github.com/RazrFalcon/resvg
cd resvg
cargo package --list
cd 
cargo package --list --manifest-path /tmp/resvg/Cargo.toml

Possible Solution(s)

No response

Notes

No response

Version

`rustc 1.59.0 (9d1b2106e 2022-02-23)` and `cargo 1.59.0 (49d8809dc 2022-02-10)`

I believe the root cause is that on macOS /tmp symlinks to /private/tmp, but Cargo does not handle symlinks and path canonicalization well for many places.

Repository::workdir() from git2 resolves the symlink whereas Package::manifest_path() does not. That results in a failure from if let Ok(status) = repo.status_file(path).

cargo/src/cargo/ops/cargo_package.rs

Lines 405 to 430 in b816d82

    
           let path = p.manifest_path(); 
        
           let path = path.strip_prefix(workdir).unwrap_or(path); 
        
           if let Ok(status) = repo.status_file(path) { 
        
               if (status & git2::Status::IGNORED).is_empty() { 
        
                   debug!( 
        
                       "found (git) Cargo.toml at {:?} in workdir {:?}", 
        
                       path, workdir 
        
                   ); 
        
                   let path_in_vcs = path 
        
                       .parent() 
        
                       .and_then(|p| p.to_str()) 
        
                       .unwrap_or("") 
        
                       .replace("\\", "/"); 
        
                   return Ok(Some(VcsInfo { 
        
                       git: git(p, src_files, &repo)?, 
        
                       path_in_vcs, 
        
                   })); 
        
               } 
        
           } 
        
           config.shell().verbose(|shell| { 
        
               shell.warn(format!( 
        
                   "No (git) Cargo.toml found at `{}` in workdir `{}`", 
        
                   path.display(), 
        
                   workdir.display() 
        
               )) 
        
           })?;

~~Although Cargo is a bit conservative on symlink resolving (see this), personally I feel like in this particular case it might be worth doing that.~~ See #10478 (comment)

@weihanglo

Actually we run cargo package --list --allow-dirty in maturin:
https://github.com/PyO3/maturin/blob/95090d027aeca1fae80c0c9fe19ae18f803db8a3/src/source_distribution.rs#L134

so I think the code in check_repo_state function referenced by @weihanglo isn't even runned.

Add a println!("{:?}", src_files); after

cargo/src/cargo/ops/cargo_package.rs

Line 97 in b816d82

let src_files = src.list_files(pkg)?;

prints: [].

I think the culprit is this code

cargo/src/cargo/sources/path.rs

Lines 293 to 298 in b816d82

    
           // Filter out files blatantly outside this package. This is helped a 
        
           // bit above via the `pathspec` function call, but we need to filter 
        
           // the entries in the index as well. 
        
           if !file_path.starts_with(pkg_path) { 
        
               continue; 
        
           }

for example, when file_path = /private/tmp/resvg/version-bump.md and pkg_path = /tmp/resvg, apparently file_path is canonicalized while pkg_path isn't.

@messense

@messense you're right. What I found is the result, not the cause. Sorry about the misleading.

BTW, I found it by making cargo package verbose.

$ cd / && cargo package --list --manifest-path /tmp/a-project/Cargo.toml --verbose
warning: No (git) Cargo.toml found at `/tmp/a-project/Cargo.toml` in workdir `/private/tmp/a-project/`
Cargo.lock

diff --git a/src/cargo/ops/cargo_package.rs b/src/cargo/ops/cargo_package.rs
index ab1a6180c..b2b6ce586 100644
--- a/src/cargo/ops/cargo_package.rs
+++ b/src/cargo/ops/cargo_package.rs
@@ -204,7 +204,7 @@ fn build_ar_list(
     vcs_info: Option<VcsInfo>,
 ) -> CargoResult<Vec<ArchiveFile>> {
     let mut result = Vec::new();
-    let root = pkg.root();
+    let root = pkg.root().canonicalize()?;
     for src_file in src_files {
         let rel_path = src_file.strip_prefix(&root)?.to_path_buf();
         check_filename(&rel_path, &mut ws.config().shell())?;
diff --git a/src/cargo/sources/path.rs b/src/cargo/sources/path.rs
index c0cdd79a9..a61204bfc 100644
--- a/src/cargo/sources/path.rs
+++ b/src/cargo/sources/path.rs
@@ -107,7 +107,8 @@ impl<'cfg> PathSource<'cfg> {
     }

     fn _list_files(&self, pkg: &Package) -> CargoResult<Vec<PathBuf>> {
-        let root = pkg.root();
+        let root = pkg.root().canonicalize()?;
+        let root = root.as_ref();
         let no_include_option = pkg.manifest().include().is_empty();
         let git_repo = if no_include_option {
             self.discover_git_repo(root)?
@@ -229,7 +230,8 @@ impl<'cfg> PathSource<'cfg> {
         let root = repo
             .workdir()
             .ok_or_else(|| anyhow::format_err!("can't list files on a bare repository"))?;
-        let pkg_path = pkg.root();
+        let pkg_path = pkg.root().canonicalize()?;
+        let pkg_path: &Path = pkg_path.as_ref();

         let mut ret = Vec::<PathBuf>::new();

So this patch fixes the problem, but I'm not sure it is the right fix since pkg.root() is used in a lot of places.

This has been fixed by switching to gitoxide for file listing (#13592). I've run the MRE and no issue in 1.79.0 and after, so close as resolved.

If there is anything I missed, let us know.

Navigation Menu

Repository navigation

`cargo package --list --manifest-path` in a workspace prints only Cargo.lock on macos #10478

Problem

Steps

Possible Solution(s)

Notes

Version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	let path = p.manifest_path();
	let path = path.strip_prefix(workdir).unwrap_or(path);
	if let Ok(status) = repo.status_file(path) {
	if (status & git2::Status::IGNORED).is_empty() {
	debug!(
	"found (git) Cargo.toml at {:?} in workdir {:?}",
	path, workdir
	);
	let path_in_vcs = path
	.parent()
	.and_then(\|p\| p.to_str())
	.unwrap_or("")
	.replace("\\", "/");
	return Ok(Some(VcsInfo {
	git: git(p, src_files, &repo)?,
	path_in_vcs,
	}));
	}
	}
	config.shell().verbose(\|shell\| {
	shell.warn(format!(
	"No (git) Cargo.toml found at `{}` in workdir `{}`",
	path.display(),
	workdir.display()
	))
	})?;

	// Filter out files blatantly outside this package. This is helped a
	// bit above via the `pathspec` function call, but we need to filter
	// the entries in the index as well.
	if !file_path.starts_with(pkg_path) {
	continue;
	}

cargo package --list --manifest-path in a workspace prints only Cargo.lock on macos #10478

Description

Problem

Steps

Possible Solution(s)

Notes

Version

Activity

weihanglo commented on Mar 14, 2022

messense commented on Mar 14, 2022

messense commented on Mar 14, 2022

messense commented on Mar 14, 2022

weihanglo commented on Mar 14, 2022

messense commented on Mar 15, 2022

weihanglo commented on Dec 29, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`cargo package --list --manifest-path` in a workspace prints only Cargo.lock on macos #10478