You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Consider integrating vals.ai after open sourcing #574
Closing as not planned. This was an open question with no concrete proposal. Since then the project has dropped its external eval integrations: LangSmith was removed in #171, the reporter/telemetry pipeline was deleted in #1174 (only the none provider remains), and paid benchmark CI was removed in #1131. Vals Smith builds benchmarks from repositories on its own platform, so it can be tried against the now-public repo without any Ultrafuzz changes. Please reopen with a specific proposal, for example using Vals-generated benchmarks as an eval suite source, if it turns out to be useful.
Activity
aviggiano commentedon Oct 2, 2026
Closing as not planned. This was an open question with no concrete proposal. Since then the project has dropped its external eval integrations: LangSmith was removed in #171, the reporter/telemetry pipeline was deleted in #1174 (only the
noneprovider remains), and paid benchmark CI was removed in #1131. Vals Smith builds benchmarks from repositories on its own platform, so it can be tried against the now-public repo without any Ultrafuzz changes. Please reopen with a specific proposal, for example using Vals-generated benchmarks as an eval suite source, if it turns out to be useful.