bzlmod example fails to build on pip install error when building whls from sdists

🐞 bug report

Affected Rule

examples/bzlmod: //:all_data_requirements

Is this a regression?

Yes, the previous version in which this bug was not present was: commit e682cd0

Description

Seems like PR #2091 breaks the bzlmod example on mac arm64.

❯ ( cd examples/bzlmod && bazel build //:all_data_requirements )
INFO: Repository rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d instantiated at:
  <builtin>: in <toplevel>
Repository rule whl_library defined at:
  /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_library.bzl:451:30: in <toplevel>
   
  ~~~~  snip ~~~~~~

===== stderr end =====
ERROR: Analysis of target '//:all_data_requirements' failed; build aborted: Analysis failed
INFO: Elapsed time: 2.092s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

🔬 Minimal Reproduction

NOTE: commit 6f9082f was isolated via git bisect

git bisect output

❯ git bisect good
6f9082fda020a9970d624e54ad3d964d64957f21 is the first bad commit
commit 6f9082fda020a9970d624e54ad3d964d64957f21
Author: Chris Chua <chris.sirhc@gmail.com>
Date:   Fri Jul 26 11:55:03 2024 +0800

    fix: use downloaded archive in sdist (#2091)

    Before this PR the extra arguments added by the `experimental_index_url`
    code paths were not used and the `sdist` was being redownloaded every
    time
    we would build from `sdist`.

    This PR fixes the code to not ignore the extra args added in

    https://github.com/bazelbuild/rules_python/pull/2091/files#diff-c007ed21502bf8ea19b98b3f1b402e7071615f8520e4291b00a71bca2cd451e8R231

    Fixes #2090

    ---------

    Co-authored-by: aignas <240938+aignas@users.noreply.github.com>

 CHANGELOG.md                        | 3 +++
 python/private/pypi/whl_library.bzl | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

git clone https://github.com/bazelbuild/rules_python
git checkout 6f9082fda020a9970d624e54ad3d964d64957f21
cd rules_python/examples/bzlmod
bazel build //:all_data_requirements

===== stderr end =====
ERROR: Analysis of target '//entry_points/tests:pylint_report' failed; build aborted: Analysis failed
INFO: Elapsed time: 1.799s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

Validate that previous version of example works:

git checkout 59bb4a88783cca738394cb3cb91bc8243df5e18d
bazel build //:all_data_requirements

INFO: Analyzed 53 targets (270 packages loaded, 23858 targets configured).
INFO: Found 53 targets...
INFO: Elapsed time: 16.998s, Critical Path: 2.83s
INFO: 165 processes: 160 internal, 5 darwin-sandbox.
INFO: Build completed successfully, 165 total actions

🔥 Exception or Error

full stack trace

ERROR: An error occurred during the fetch of repository 'rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_library.bzl", line 243, column 35, in _whl_library_impl
                repo_utils.execute_checked(
        File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/repo_utils.bzl", line 204, column 29, in _execute_checked
                return _execute_internal(fail_on_error = True, *args, **kwargs)
        File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/repo_utils.bzl", line 135, column 20, in _execute_internal
                logger.fail((
        File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/repo_utils.bzl", line 85, column 39, in lambda
                fail = lambda message_cb: _log(-1, "FAIL", message_cb, fail),
        File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/repo_utils.bzl", line 75, column 16, in _log
                printer("\nrules_python:{} {}:".format(
Error in fail:
rules_python:whl_library(@@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d) FAIL: repo.execute: whl_library.BuildWheelFromSource(rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d, lazy-object-proxy==1.10.0): end: failure:
  command: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python -m python.private.pypi.whl_installer.wheel_installer --requirement lazy-object-proxy==1.10.0 --isolated --extra_pip_args "{\"arg\":[\"--no-index\",\"--find-links\",\".\"]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: <default: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d>
  timeout: 600
  environment:
PYTHONPATH="/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__build:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__click:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__colorama:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__importlib_metadata:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__installer:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__more_itertools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__packaging:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pep517:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip_tools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pyproject_hooks:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__setuptools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__tomli:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__wheel:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -isystem /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/include/python3.9"
===== stdout start =====
Looking in links: .
Processing ./lazy-object-proxy-1.10.0.tar.gz (from -r /var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h (line 1))
  File was already downloaded /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d/lazy-object-proxy-1.10.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
===== stdout end =====
===== stderr start =====
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      Looking in links: .
      ERROR: Could not find a version that satisfies the requirement setuptools>=64 (from versions: none)
      ERROR: No matching distribution found for setuptools>=64
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 205, in <module>
    main()
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 190, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '--no-index', '--find-links', '.', '-r', '/var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h']' returned non-zero exit status 1.
===== stderr end =====
ERROR: no such package '@@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d//':
rules_python:whl_library(@@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d) FAIL: repo.execute: whl_library.BuildWheelFromSource(rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d, lazy-object-proxy==1.10.0): end: failure:
  command: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python -m python.private.pypi.whl_installer.wheel_installer --requirement lazy-object-proxy==1.10.0 --isolated --extra_pip_args "{\"arg\":[\"--no-index\",\"--find-links\",\".\"]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: <default: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d>
  timeout: 600
  environment:
PYTHONPATH="/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__build:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__click:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__colorama:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__importlib_metadata:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__installer:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__more_itertools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__packaging:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pep517:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip_tools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pyproject_hooks:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__setuptools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__tomli:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__wheel:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -isystem /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/include/python3.9"
===== stdout start =====
Looking in links: .
Processing ./lazy-object-proxy-1.10.0.tar.gz (from -r /var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h (line 1))
  File was already downloaded /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d/lazy-object-proxy-1.10.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
===== stdout end =====
===== stderr start =====
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      Looking in links: .
      ERROR: Could not find a version that satisfies the requirement setuptools>=64 (from versions: none)
      ERROR: No matching distribution found for setuptools>=64
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 205, in <module>
    main()
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 190, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '--no-index', '--find-links', '.', '-r', '/var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h']' returned non-zero exit status 1.
===== stderr end =====
ERROR: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip/lazy_object_proxy/BUILD.bazel:153:6: @@rules_python~~pip~pip//lazy_object_proxy:data depends on @@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d//:data in repository @@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d which failed to fetch. no such package '@@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d//':
rules_python:whl_library(@@rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d) FAIL: repo.execute: whl_library.BuildWheelFromSource(rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d, lazy-object-proxy==1.10.0): end: failure:
  command: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python -m python.private.pypi.whl_installer.wheel_installer --requirement lazy-object-proxy==1.10.0 --isolated --extra_pip_args "{\"arg\":[\"--no-index\",\"--find-links\",\".\"]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: <default: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d>
  timeout: 600
  environment:
PYTHONPATH="/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__build:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__click:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__colorama:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__importlib_metadata:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__installer:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__more_itertools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__packaging:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pep517:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pip_tools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__pyproject_hooks:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__setuptools:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__tomli:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__wheel:/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -isystem /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/include/python3.9"
===== stdout start =====
Looking in links: .
Processing ./lazy-object-proxy-1.10.0.tar.gz (from -r /var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h (line 1))
  File was already downloaded /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d/lazy-object-proxy-1.10.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
===== stdout end =====
===== stderr start =====
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      Looking in links: .
      ERROR: Could not find a version that satisfies the requirement setuptools>=64 (from versions: none)
      ERROR: No matching distribution found for setuptools>=64
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 205, in <module>
    main()
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 190, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_aarch64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '--no-index', '--find-links', '.', '-r', '/var/folders/yw/jyw6crv53631v327332qwrgm0000gp/T/tmp3ugtq99h']' returned non-zero exit status 1.
===== stderr end =====
ERROR: Analysis of target '//:all_data_requirements' failed; build aborted: Analysis failed
INFO: Elapsed time: 2.092s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

🌍 Your Environment

Operating System:

  
❯ uname -a
Darwin TY779MHFP2.TUBI 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64

Output of bazel version:

  
❯ bazel version
Bazelisk version: development
Build label: 7.2.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Jun 25 15:53:57 2024 (1719330837)
Build timestamp: 1719330837
Build timestamp as int: 1719330837

Rules_python version:

  
HEAD branch as of commit 6f9082fda020a9970d624e54ad3d964d64957f21

Anything else relevant?

I tried manually installing lazy-object-proxy in the rules_python python interpreter, and was able to successfully install only by removing the --no-index option to pip. Not sure why --no-index tells pip to not use the installed setuptools (which indeed has version >= 64)

❯ tar xzf $(bazel info output_base)/external/rules_python~~pip~pip_39_lazy_object_proxy_sdist_78247b6d/lazy-object-proxy-1.10.0.tar.gz
❯ cd lazy-object-proxy-1.10.0
❯ $(bazel info output_base)/external/rules_python~~python~python_3_9_host/python -m pip list
Package    Version
---------- -------
pip        24.0
setuptools 69.1.0

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: /private/var/tmp/_bazel_slayland/96ddcf3f8cdebe38a185655c2901b33f/external/rules_python~~python~python_3_9_host/python -m pip install --upgrade pip

Fails to install with

$(bazel info output_base)/external/rules_python~~python~python_3_9_host/python -m pip install --isolated --no-deps --find-links . --no-index .

Works with

$(bazel info output_base)/external/rules_python~~python~python_3_9_host/python -m pip install --isolated --no-deps --find-links . .

The fix for this may be to upgrade to requirements where lazy-object proxy is available for arm macs.

This is showing how the experimental requirements feature is working and unfortunately we have no arm macs for CI to verify that everything works.

Hm I'm seeing this issue too, but it seems to be pointing to the issue where setuptools is not available to be installed.
It's building from the sdist, so I'd not expect it to need an arm macs build of lazy-proxy-obj .

This issue was hidden before 6f9082f as sdist were installed via pip (which would then automatically install setuptools as required).
This looks like the issue I was hitting in #2090 (comment) but is easier to reproduce.

A minimum reproduction would be a package that only has a sdist and no wheel that is compatible with the current host platform where you're testing from, and a pyproject.toml containing build requires like:
https://github.com/ionelmc/python-lazy-object-proxy/blob/aa46e846fa2eb7cb845a07e122f2984267551098/pyproject.toml#L1-L5

[build-system]
requires = [
    "setuptools>=64",
]

In case it's helpful, you may want to try reproducing with https://pypi.org/project/pytest-shard/ . I saw this issue on occurring on that package.

I saw:

Full logs


===== stdout start =====
--
  | Looking in links: .
  | Processing ./pytest-shard-0.1.2.tar.gz (from -r /tmp/tmpczrz__vl (line 1))
  | File was already downloaded /var/cache/bazel/output_base/external/rules_python~~pip~third_party_python_base_39_pytest_shard_sdist_b86a967f/pytest-shard-0.1.2.tar.gz
  | Installing build dependencies: started
  | Installing build dependencies: finished with status 'error'
  | ===== stdout end =====
  | ===== stderr start =====
  | error: subprocess-exited-with-error
  |  
  | × pip subprocess to install build dependencies did not run successfully.
  | │ exit code: 1
  | ╰─> [3 lines of output]
  | Looking in links: .
  | ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
  | ERROR: No matching distribution found for setuptools>=40.8.0
  | [end of output]
  |  
  | note: This error originates from a subprocess, and is likely not a problem with pip.
  | error: subprocess-exited-with-error
  |  
  | × pip subprocess to install build dependencies did not run successfully.
  | │ exit code: 1
  | ╰─> See above for output.
  |  
  | note: This error originates from a subprocess, and is likely not a problem with pip.
  | Traceback (most recent call last):
  | File "/var/cache/bazel/output_base/external/rules_python~~python~python_3_9_x86_64-unknown-linux-gnu/lib/python3.9/runpy.py", line 197, in _run_module_as_main
  | return _run_code(code, main_globals, None,
  | File "/var/cache/bazel/output_base/external/rules_python~~python~python_3_9_x86_64-unknown-linux-gnu/lib/python3.9/runpy.py", line 87, in _run_code
  | exec(code, run_globals)
  | File "/var/cache/bazel/output_base/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 205, in 
  | main()
  | File "/var/cache/bazel/output_base/external/rules_python~/python/private/pypi/whl_installer/wheel_installer.py", line 190, in main
  | subprocess.run(pip_args, check=True, env=env)
  | File "/var/cache/bazel/output_base/external/rules_python~~python~python_3_9_x86_64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 528, in run
  | raise CalledProcessError(retcode, process.args,
  | subprocess.CalledProcessError: Command '['/var/cache/bazel/output_base/external/rules_python~~python~python_3_9_host/python', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '--no-index', '--find-links', '.', '-r', '/tmp/tmpczrz__vl']' returned non-zero exit status 1.
  | ===== stderr end =====

Seeing same error

@mark-thm

Note, that sdist building in general is not hermetic, so your success may vary. It would be best to push wheels for the said sdists to a private artifact registry so that one does not need to build sdist within rules_python.

I think there are other potential solutions to unblock people (aside from using #2126 as a patch):

Merge the removal of --no-index in the sdist building, which is the PR fix(whl_library): remove --no-index and add --no-build-isolation when build sdist #2126.
As @mark-thm suggested, we should also pass --no-build-isolation, because we are constructing our own PYTHONPATH and the setuptools would be then taken from rules_python maintaining hermeticity.
@mark-thm has also suggested parsing the pyproject.toml to add build dependencies automatically to the PYTHONPATH, however, since this is done at the repository_rule level, I would like to not move forward here. It would be better to invest time in making the bottom two options work.
Support pip.override passing a whl file into the pip hub repository. That may still bring questions how to wire that up with wheel file discovery during requirements.txt lock file generation, so this might be medium to high complexity.
Implement a way to pass a BUILD.bazel file contents to pip.parse for the sdists that do not build - we could use a simple http_archive to extract the wheel and ask the user to craft their own BUILD.bazel file by hand. This could be medium difficulty, as the main difficulty is to craft the BUILD.bazel file which would be offloaded to the users of rules_python. I suspect that sometimes a simple genrule using pip as a cmd could do the trick and then we could use whl_filegroup or something similar to extract the files and pass them to py_library. We also have a rule from rules_pycross for creating a py_library like thing from a whl file itself, so that could be also a good way to make use of the code. It is probably suboptimal in terms of performance, but potentially much more maintainable.

The last two options would be my prefered ones because that makes rules_python more useful.

re:

It would be best to push wheels for the said sdists to a private artifact registry so that one does not need to build sdist within rules_python.

I think I have a slightly different take here, which is that we'd like to use Bazel to hermetically build all the sdists and not depend on pre-built wheels so that we can manage supply chain build concerns. Even if we were to manage a private artifact repository, we'd still want to use Bazel to run the sdist builds and publishing steps since we use Bazel to build/orchestrate the rest of our infrastructure.

At any rate, I think the community would benefit from guidance on how to deal with sdists that need to be built, especially where upstream projects are unwilling or unable to take on the burden of producing wheels for different architectures/platforms.

@ewianda

Thank you all for the work and thoughts on this! If you'll forgive me for adding some unsolicited opinions, I have some concerns about the last two suggestions.

As an end user who wants things to be as simple as possible, my biggest sticking point with the last two of @ewianda 's solutions is that it seems like the maintenance/complexity is passed down to the python developer. A python developer who is used to the fact that 3 9s percent of the time they just need to run pip install foo (or add a single line to a pyproject.toml, Pipfile, or requirements.txt file) to get things to working for their project.

From a hermetic build standpoint, using pip to download and build sdists can be a problem. From a bazel adoption standpoint, where users want to reliably build applications in one of the most popular languages, adding non ergonomic build configuration steps for things python devs don't care about is annoying and time consuming in the least (or require dedicated hand holding from a build engineer).

Adopting bazel is not an easy paradigm shift for people to begin with, so anything to dull the edges here would be preferable. Otherwise, expect more threads like these.

(context: I'm doing a POC migration for a large python/scala/java/terraform monorepo and evaluating bazel/buck2/pants. For bazel, I'm getting stuck the most with getting ergonomic python builds for many independent py projects in the monorepo, each with their own currently working pyproject.toml /pip lock files. As things stand, I worry that my python devs will revolt if I push further complexity on them! 🤣)

Thanks for the message. I am happy that bazel is getting traction in the python world and I am happy for you that you are looking into ways of adopting bazel in your organization. By the way, from the list I posted previously, only the first solution comes from @ewianda, so let me take the blame here. :) My reasoning with the last two solutions was to give the build engineer full power to have full hermeticity as opposed to relying on the non-hermetic repository rules. Not using wheels at all and having everything just work could be doable, but that may be outside the immediate or even medium-term future. For that to be viable, someone would have to port/reimplement rules_pycross within rules_python. Since this is something that only occurs when the experimental feature is used, the priority of fixing sdists may be low until the rest of the code is stabilized. For the time being I think adding --no-build-isolation and removing --no-index us probably the best course of action - contributions are welcome here and thanks to @ewianda for submitting the PR! P.S. it's up to people to vent online about bazel. There will always be people who don't like X or Y and I certainly think the maintainers of this project are not responsible for devs in your company not liking bazel - everyone is free to have their opinions. :) I am always happy when people contribute to the project by letting the maintainers know about existing issues and or submitting PRs, but those threads on reddit or elsewhere do not move the needle here. :)

…

On 21 August 2024 19:57:47 EEST, steve layland ***@***.***> wrote: Thank you all for the work and thoughts on this! If you'll forgive me for adding some unsolicited opinions, I have some concerns about the last two suggestions. As an end user who wants things to be as simple as possible, my biggest sticking point with the last two of @ewianda 's solutions is that it seems like the maintenance/complexity is passed down to the python developer. A python developer who is used to the fact that 3 9s percent of the time they just need to run `pip install foo` (or add a single line to a pyproject.toml, Pipfile, or requirements.txt file) to get things to working for their project. From a hermetic build standpoint, using pip to download and build sdists can be a problem. From a bazel adoption standpoint, where users want to reliably build applications in one of the most popular languages, adding non ergonomic build configuration steps for things python devs don't care about is annoying and time consuming in the least (or require dedicated hand holding from a build engineer). Adopting bazel is not an easy paradigm shift for people to begin with, so anything to dull the edges here would be preferable. Otherwise, expect more [threads](https://news.ycombinator.com/item?id=38493940) like [these](https://www.reddit.com/r/devops/s/bdP3bW8fVK). (context: I'm doing a POC migration for a large python/scala/java/terraform monorepo and evaluating bazel/buck2/pants. For bazel, I'm getting stuck the most with getting ergonomic python builds for many independent py projects in the monorepo, each with their own currently working pyproject.toml /pip lock files. As things stand, I worry that my python devs will revolt if I push further complexity on them! 🤣) -- Reply to this email directly or view it on GitHub: #2118 (comment) You are receiving this because you commented. Message ID: ***@***.***>

@rockpunk

For the time being I think adding --no-build-isolation and removing --no-index

I will add this to the PR. Unfortunately, I already pushed the wheels of the two offending packages to our private repository, so I can't verify that adding --no-build-isolation actually works.

Credit for this solution should actually go to @rockpunk

:) I am always happy when people contribute to the project by letting the maintainers know about existing issues and or submitting PRs, but those threads on reddit or elsewhere do not move the needle here. :)

Fair point, you're totally right; i shouldn't have brought in external noise. That said, in my research these past couple of months, the pervading sentiment that I've seen is that Bazel and python are still unnecessarily complex from a developer POV and I want to encourage anything that changes that perception. Thanks again for the hard work!

fix(whl_library): remove `--no-index` and add `--no-build-isolation` …

Navigation Menu

Repository navigation

bzlmod example fails to build on pip install error when building whls from sdists #2118

🐞 bug report

Affected Rule

Is this a regression?

Description

🔬 Minimal Reproduction

🔥 Exception or Error

🌍 Your Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

bzlmod example fails to build on pip install error when building whls from sdists #2118

Description

🐞 bug report

Affected Rule

Is this a regression?

Description

🔬 Minimal Reproduction

🔥 Exception or Error

🌍 Your Environment

Activity

aignas commented on Aug 11, 2024

chrisirhc commented on Aug 12, 2024

ewianda commented on Aug 18, 2024

aignas commented on Aug 21, 2024

mark-thm commented on Aug 21, 2024

rockpunk commented on Aug 21, 2024

aignas commented on Aug 21, 2024

ewianda commented on Aug 21, 2024

rockpunk commented on Aug 21, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions