`--downloader_config` doesn't appear to be respected?

[dougthor42]

Description of the bug:

It seems like --downloader_config isn't being used. I've set it to block all access block .* but a project is still able to download things from BCR and GitHub and other places.

The documentation for --downloader_config is somewhat lacking, so I'm not 100% sure I'm doing things correctly. It's entirely possible that either my expectations are incorrect or I'm doing something wrong.

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Steps:

1. Create a config file that blocks everything block .*
2. Use that config file: --downloader_config=block_all.cfg

Expected result:

Nothing gets downloaded, all builds fail.

Actual Result:

Builds still succeed.

Minimal example:

$ ls -lah
total 40K
drwxr-x--- 2 dthor primarygroup 4.0K Apr 13 23:33 .
drwxr-x--- 3 dthor primarygroup 4.0K Apr 13 23:24 ..
-rw-r----- 1 dthor primarygroup    6 Apr 13 23:27 .bazelversion
-rw-r----- 1 dthor primarygroup    9 Apr 13 23:28 block_all.cfg
-rw-r----- 1 dthor primarygroup   79 Apr 13 23:32 MODULE.bazel
$
$ cat .bazelversion 
8.3.1
$ cat block_all.cfg 
block .*
$ cat MODULE.bazel
module(name = "example")

bazel_dep(name = "rules_license", version = "0.0.7")
$
$ bazel clean --expunge
$ bazel build --downloader_config=block_all.cfg @rules_license//:standard_package
Starting local Bazel server (8.3.1) and connecting to it...
WARNING: For repository 'rules_license', the root module requires module version rules_license@0.0.7, but got rules_license@1.0.0 in the resolved dependency graph. Please update the version in your MODULE.bazel or set --check_direct_dependencies=off
INFO: Analyzed target @@rules_license+//:standard_package (6 packages loaded, 17 targets configured).
INFO: Found 1 target...
Target @@rules_license+//:standard_package up-to-date (nothing to build)
INFO: Elapsed time: 3.970s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

Which operating system are you running Bazel on?

Debian

What is the output of bazel info release?

release 8.3.1

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

N/A

What's the output of git remote get-url origin; git rev-parse HEAD ?

N/A

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

Tried but it's not really something that is easily found via bisect because of caches.

$ bazelisk-linux-amd64_v1.26.0 --bisect=8.0.0..HEAD build --downloader_config=block_all.cfg @rules_license//:standard_package
                                                                                                                                                                                                                                                                                                                                                                                                                                            
                                                                                                           
--- Getting the list of commits between 8.0.0 and HEAD                                                     
                                                                                                                                                                                                                                                                                                                                                                                                                                            
The old Bazel commit is not an ancestor of the new Bazel commit, overriding the old Bazel commit to the merge base commit d0285222b0060847d815d87d6757ca22e43ebd93                                                    
Found 4493 commits between (d0285222b0060847d815d87d6757ca22e43ebd93, HEAD]

...

first bad commit not found, every commit succeeded.

Have you found anything relevant by searching the web?

I asked in the internal Google Bazel Users chat /shrug.

Any other information, logs, or outputs that you want to share?

No response

[keith]

the block syntax is block * not block .*

after your bazel clean --expunge you also want to pass --repository_cache= to make sure it's not pulling those archives from a cache. changing the config doesn't bust the cache of all the archives you've previously downloaded or downloaded from other projects. (in the past i've run a check like this on CI to make sure no one's local changes to this file gets in because of this issue too)

[dougthor42]

🤦 thank you!

Ah, upon really reading the docs I see that now:

This file consists of lines, each of which starts with a directive (allow, block or rewrite) followed by either a host name (for allow and block) or two patterns, one to match against, and one to use as a substitute URL

Reworded:

block and allow take host names.
rewrite takes two regex patterns separated by a space.

If you don't mind, I'll submit a PR to make the docs more clear.

Followup questions so that I can include the answers in the doc updates:

1. are comments supported in the config file?
2. is it possible to block just some paths of a given host? eg github.com/bazel-contrib/rules_python
   • if not, I assume the way to do this is a rewrite + block?:

     rewrite .*/github.com/bazel-contrib/rules_python/.* blocked_host.com
     block blocked_host.com
     

[keith]

doc updates with example would be great! I think this format has always been underdocumented. so much so that i don't know the answer to your questions either, i'd try it out to see!