I build and audit evaluations for AI systems that act in real environments. My work sits at the boundary between model behavior, agent harnesses, tools, sandboxes, verifiers, and the infrastructure that can quietly change what a benchmark result means.
I work in public through Hive Fidelity AI, an independent evaluation-science lab. We are not for hire. We are open to non-paid open-source work, volunteer third-party safety evaluation, and research collaboration.
Research notes · Hive Fidelity AI · LinkedIn
- Agentic evaluation: separating model capability from prompts, tools, memory, orchestration, environment behavior, and grading.
- Evaluation validity: verifier falsification, reward-hacking resistance, task repair, contamination controls, and evidence-preserving result reconciliation.
- Safety and cyber: prompt-injection, harmful-behavior, mental-health safety, and controlled cyber-capability evaluations.
- Open evaluation infrastructure: OpenHands, Harbor, NeMo Gym, Inspect Evals, SWE-bench, τ²/τ³, and reproducible agent trajectories.
|
I convert surplus coding-agent usage into open-source maintenance. When one of my subscriptions has capacity left, I dispatch a Usage-Surplus OSS Agent into an evaluation or agent-infrastructure repository to reproduce a real bug, implement a fix, run the project's native verification, and prepare an upstream contribution. I choose the missions and own the submissions. Agent authorship is disclosed in the commit and PR history, and every patch is reviewed by a human before it is marked ready for maintainer review. Current field agents: Claude and Codex, with other coding agents joining as surplus capacity permits. Request a deployment: kirsten@reinainblood.dev Drafted by Special Agent Claude of the Usage-Surplus OSS Program. No tokens left behind. |
Special Agent, Upstream Affairs |
These are the public contributions currently in review. Several were developed with Codex or Claude Code under my direction; agent involvement is disclosed in the PR body, and I am responsible for the submitted code and evidence.
My NeMo Gym work is developed and reviewed in a public fork before any upstream submission decision. The current integration set includes real execution evidence, pinned source data, deterministic preparation, and benchmark-specific verification rather than score-import shims.
| Integration | Public work |
|---|---|
| Modal sandbox provider | reinainblood/Gym#1 — V2 sandbox lifecycle, file transfer, networking controls, registry integration, and cleanup |
| InjecAgent | reinainblood/Gym#2 — indirect prompt-injection benchmark adapter |
| HarmBench | reinainblood/Gym#3 — DirectRequest protocol with target and classifier separation |
| Safe-Child-LLM | reinainblood/Gym#4 — human-evaluation workflow across developmental prompt sets |
| FACTS Grounding v2 | reinainblood/Gym#5 — public set, official two-judge protocol, calibration, and reproducible reports |
| VERA-MH | reinainblood/Gym#6 — multi-turn mental-health safety evaluation with the official rubric judge |
Related public benchmark surfaces already in the upstream ecosystems include τ² in NeMo Gym and τ³ in Harbor. I work on the agent, environment, calibration, and verifier boundaries around both.
- kimi-pi — a Pi fork used for evaluation-oriented agent work, including provider fault handling and a coding-agent resource ledger.
- dsh-codex-connect — public DSH/Codex connector work, including Desktop context overrides and the tested 4.25–4.28 runtime adaptation branches.
- Gym — public NeMo Gym integration fork and review boundary for reusable evaluation packages.
I do not treat a trajectory as a successful outcome, a deployed service as a completed evaluation, or a product stack leaderboard as a pure model ranking. Public results should carry the agent and harness configuration, task and verifier provenance, invalid-infrastructure accounting, complete trajectories, and enough evidence for someone else to disagree productively.
That is the work. No animated typing cursor required.