Skip to content

feat(browsecomp): support runtime-selectable web research providers #4709

Open

Description

@cwing-nvidia

Background

BrowseComp users may need different web research providers because of availability, credentials, cost, rate limits, regional access, or evaluation requirements.

Today, Tavily and Exa are implemented directly inside the BrowseComp resources server. Adding another provider requires extending that server’s configuration, dispatch logic, response normalization, metrics, caching, and tests.

PR #3503 demonstrates this for Brave, while the closed PR #3460 shows how substantially the integration can grow for another provider.

This coupling makes provider experimentation harder and risks provider-specific behavior becoming mixed with the benchmark’s stable responsibilities, such as contamination filtering, result budgets, workspace handling, and verification.

Users would benefit from being able to choose a supported web research provider when configuring a run without modifying BrowseComp’s agent, task data, or core implementation.

Description

Define a supported way for BrowseComp’s web research provider to be selected and configured at runtime while retaining a stable search and browse experience for the agent.

The resulting design should:

  • Keep benchmark-owned behavior consistent across providers.
  • Keep provider credentials and deployment details outside benchmark data.
  • Make provider identity and configuration visible in rollout metadata and metrics.
  • Support both current providers and future integrations such as Brave.

Activity

  1. marcromeyn commented on Oct 28, 2026

    @marcromeyn
    Contributor

    @ritugala, routing this to you as BrowseComp owner. Context: #3503 (Brave) is waiting on this decision, and #3460 (You.com, closed) shows how large each hard-coded provider gets. Would you prefer a provider interface first, with Brave as its first provider, or landing #3503 and refactoring afterwards?

  2. added
    featureNew capabilities, enhancements, or enablement work
    area:environmentIndividual environments, benchmarks, verifiers, and environment-specific resources servers
    on Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Environment & Benchmarksarea:environmentIndividual environments, benchmarks, verifiers, and environment-specific resources serversfeatureNew capabilities, enhancements, or enablement work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions