Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions benchmarks/browsecomp/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,19 @@
# Run instructions

BrowseComp uses Tavily by default. To use Brave, compose
`benchmarks/browsecomp/provider_brave.yaml` with the base benchmark config.

The Brave provider uses Brave's public [LLM Context API](https://api-dashboard.search.brave.com/documentation/services/llm-context)
for Search. It persists the returned URL context per session under `/tmp` so
Browse can serve matching URLs without issuing another provider request.

Add the provider credentials to `env.yaml`:

```yaml
brave:
api_key: <Brave Search API subscription token>
```

1. Update your env.yaml file with the necessary keys and configs. You can leave `exclude_domains_file_path` as a path to an empty json.
```yaml
browsecomp_benchmark_resources_server:
Expand Down
23 changes: 23 additions & 0 deletions benchmarks/browsecomp/provider_brave.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# BrowseComp search and browse use Brave's public LLM Context API directly.
browsecomp_benchmark_resources_server:
resources_servers:
browsecomp_advanced_harness:
search_provider: brave
brave:
api_key: ${brave_search_api_key}
exclude_domains_file_path: benchmarks/browsecomp/data/empty_exclude_domains.json
5 changes: 4 additions & 1 deletion resources_servers/browsecomp_advanced_harness/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ requiring multi-hop investigation through search and page-level browsing.

## Overview

This resources server provides two browsing tools backed by [Tavily](https://tavily.com/):
This resources server provides two browsing tools backed by Tavily, Exa, or
Brave:

- **`search`** — parallel web search returning top results with content snippets
- **`browse`** — full-page content extraction from given URLs
Expand All @@ -18,6 +19,8 @@ Add to your `env.yaml` (or pass via CLI):

```yaml
tavily_api_key: <YOUR_TAVILY_KEY> # or a list of keys for rotation
brave:
api_key: <YOUR_BRAVE_SEARCH_API_KEY>
exclude_domains_file_path: /path/to/excluded_domains.json

judge_model_base_url: <YOUR_JUDGE_MODEL_URL>
Expand Down
Loading