Activity

1. aignas commented on Jun 16, 2026

   

   aignas

   on Jun 16, 2026

   Collaborator

   More actions

   2.0.3 should not have this bug, so this is really strange and the unit tests validate that behaviour. Did you clear all of the caches?

   If you have 2 indexes specified, it will still query, so may need to specify your default index in addition to doing extra index urls. Could you print here the MODULE.bazel setup and the headers of the requirements files?
2. dougthor42 commented on Jun 17, 2026

   

   dougthor42

   on Jun 17, 2026

   CollaboratorAuthor

   More actions

   Just tried with the official 2.0.3 instead of a git_override (ok fine technically it was a local_path_override to my clone of rules_python... 😆).

   --- a/MODULE.bazel
   +++ b/MODULE.bazel
   @@ -23,7 +23,7 @@ PYTHON_VERSION_NAME = "python_" + PYTHON_VERSION.replace(".", "_")
    
    PYNEXT_VERSION_NAME = "python_" + PYNEXT_VERSION.replace(".", "_") if PYNEXT_VERSION != None else None  # @unused
    
   -RULES_PYTHON_VERSION = "1.9.0"
   +RULES_PYTHON_VERSION = "2.0.3"
    
    RULES_RUST_VERSION = "0.70.0"

   Clearing caches (bazel clean --expunge_async) had no effect.

   $ bazel clean --expunge_async; bazel run //tools/bazel/gazelle:gazelle_python_manifest.update
   ...
   Error in wait: java.io.IOException: Error downloading [https://oauth2accesstoken@us-west2-python.pkg.dev/GCS-PROJECT/GAR-NAME/simple/] to /usr/local/google/home/dthor/.cache/bazel/_bazel_dthor/0f8c52850e7230283fc2f8033149fba2/modextwd/rules_python++pip/https___oauth2accesstoken@us_west2_python_pkg_dev_qh_lab_qh_py_simple.html: GET returned 404 Not Found

   Here are the relevant bits of MODULE.bazel:

   PYTHON_VERSION = "3.13.6"
   
   RULES_PYTHON_VERSION = "2.0.3"
   
   PRIVATE_INDEX_URL = "https://us-west2-python.pkg.dev/GCS-PROJECT/GAR-NAME/simple"
   
   EXPERIMENTAL_INDEX_URL_OVERRIDES = {
       "some_package": PRIVATE_INDEX_URL,
       ...,
   }
   
   pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
   pip.parse(
       experimental_index_url = AIRLOCK_INDEX_URL,  # basically a copy of pypi
       experimental_index_url_overrides = EXPERIMENTAL_INDEX_URL_OVERRIDES,
       experimental_requirement_cycles = {
           "mdit_py_plugins": [
               "markdown_it_py",
               "mdit_py_plugins",
           ],
       },
       hub_name = "pypi",
       python_version = PYTHON_VERSION,
       requirements_lock = "//:requirements.txt",
       target_platforms = ["{os}_{arch}"],
   )

   And our requirements.txt file:

   $ head `requirements.txt`
   # This file was autogenerated by uv via the following command:
   #    uv pip compile --python-version=3.13 --output-file=requirements.txt requirements.in
   --index-url https://oauth2accesstoken@us-python.pkg.dev/REDACTED/REDACTED/simple  # AIRLOCK_INDEX_URL
   --extra-index-url https://oauth2accesstoken@us-west2-python.pkg.dev/GCP-PROJECT/GAR-NAME/simple
   
   absl-py==2.3.0 \
       --hash=sha256:9824a48b654a306168f63e0d97714665f8490b8d89ec7bf2efc24bf67cf579b3 \
       --hash=sha256:d96fda5c884f1b22178852f30ffa85766d50b99e00775ea626c23304f582fc4f
       # via
       #   dm-tree

   Note that AIRLOCK_INDEX_URL also does not support the /simple endpoint:

   $ curl --fail-with-body -s -L --header "Authorization: Bearer $(gcloud auth print-access-token)" "${AIRLOCK_INDEX_URL}"
   
   <!DOCTYPE html>
   <html lang=en>
     <meta charset=utf-8>
     <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
     <title>Error 404 (Not Found)!!1</title>
     <style>
       ...
     </style>
     <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
     <p><b>404.</b> <ins>That’s an error.</ins>
     <p>  <ins>That’s all we know.</ins>
3. aignas commented on Jun 19, 2026

   

   aignas

   on Jun 19, 2026

   Collaborator

   More actions

   A few things:

   • You can drop the experimental_index_url from the pip.parse. Just the overrides is fine.
   • The code that should work fine in your case and not require the calls to the /simple/ endpoint is here

   It is hard to debug remotely, so please dig in that function and add more debugging to better understand why the if in there is not triggered.
4. dougthor42 commented on Jun 19, 2026

   

   dougthor42

   on Jun 19, 2026

   CollaboratorAuthor

   More actions

   TL;DR: I'm a bit of a dumb-dumb.

   We have another pip.parse & requirements lock file that includes --extra-index-url even though it doesn't need anything from the extra index. Since there are two indexes and we don't set the overrides for pip.parse, things get down to L154. And neither of our indexes support GET .../simple.

   The fix is to remove --extra-index-url from that 2nd requirements lock file.

   I believe everything is working as intended, though I wonder if it's prudent to have a fallback if GET .../simple errors out.

   ---

   It appears that _get_dist_urls is being called twice.
   On the 2nd call, index_url_overrides is empty and len(index_urls) == 2.

   ...

   Oh shit, I completely forgot that we have a 2nd pip.parse to create a hub dedicated for twine:

   pip.parse(
       experimental_index_url = AIRLOCK_INDEX_URL,
       hub_name = "pypi_twine",
       python_version = PYTHON_VERSION,
       requirements_lock = "//tools/bazel:requirements_twine.txt",
       target_platforms = ["{os}_{arch}"],
   )

   $ head tools/bazel/requirements_twine.txt
   # This file was autogenerated by uv via the following command:
   #    uv pip compile --python-version=3.13 --output-file=tools/bazel/requirements_twine.txt tools/bazel/requirements_twine.in
   --index-url https://oauth2accesstoken@us-python.pkg.dev/REDACTED/REDACTED/simple
   --extra-index-url https://oauth2accesstoken@us-west2-python.pkg.dev/GCP-PROJECT/GAR-NAME/simple
   
   cachetools==5.2.1 \
       --hash=sha256:5991bc0e08a1319bb618d3195ca5b6bc76646a49c21d55962977197b301cc1fe \
       --hash=sha256:8462eebf3a6c15d25430a8c27c56ac61340b2ecf60c9ce57afc2b97e450e47da
       # via
       #   -c tools/bazel/../../requirements.txt

   $ cat tools/bazel/requirements_twine.in
   # Requirements needed to upload Python wheels via `twine`.
   #
   # This is used as a separate rules_python pip hub so that building and
   # uploading the rust rupyle wheel doesn't take as long.
   
   --index-url https://oauth2accesstoken@us-python.pkg.dev/REDACTED/REDACTED/simple
   --extra-index-url https://oauth2accesstoken@us-west2-python.pkg.dev/GCP-PROJECT/GAR-NAME/simple
   
   # Constrain versions to match those in requirements.txt
   -c ../../requirements.txt
   
   # go/keep-sorted start
   keyrings.google-artifactregistry-auth
   twine
   # go/keep-sorted end
   

   The fix is to remove --extra-index-url from tools/bazel/requirements_twine.(in|txt).

   ---

   Also yes we can safely remove experimental_index_url = AIRLOCK_INDEX_URL.

   We can also remove our patch for whl_library.BuildWheelFromSource! I'm not 100% sure why but I'll take the win haha. (so #2640 is fixed!)
5. dougthor42

   closed this as completedon Jun 19, 2026
6. rickeylev commented on Jun 19, 2026

   

   rickeylev

   on Jun 19, 2026

   Collaborator

   More actions

   I was looking at the GAR docs, and it says it does support the simple API: https://docs.cloud.google.com/artifact-registry/docs/python

   These private repositories use ... the simple repository API, and work with installation tools like pip.

   I couldn't find anything specific about how to setup the simple API, so presumably it's supposed to just work? But clearly it's not in, at least, your case
7. dougthor42 commented on Jun 20, 2026

   

   dougthor42

   on Jun 20, 2026

   CollaboratorAuthor

   More actions

   I opened an internal bug with the Artifact Registry team. You're cc'd on it, Richard.

   ❤️React with ❤️2Reacted by Richard Levasseur and Ignas Anikevicius