Repository navigation
docs(fern): adding an evaluation section and fixing other eval references - #1740
Conversation
… rebasing issues Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
Signed-off-by: Rita Fernandes Neves <rfernandesne@nvidia.com>
sephmard
left a comment
There was a problem hiding this comment.
Approving — the core items from #1676 are addressed: EvalPlus tutorial improvements, Add a Benchmark moved to Evaluation, --num-repeats stats clarified, agent-as-unit-under-test section, environment list column cleanup, notification removed, and the new Skills page under Evaluation covers BLADE.
Two small follow-ups to track: #1743
-
BLADE definition: Both the Skills page and the BLADE tutorial jump into the workflow without first grounding the reader in what BLADE is. Suggest opening both with a short intro — what BLADE stands for, that it's a post-run analysis workflow (not a benchmark runner), and the core question it answers ("why did the score change?"). The #1676 review thread has the full suggested framing.
-
Rename "Skills" to "Benchmark Analysis": The current nav label "Skills" is abstract for a new user. "Benchmark Analysis" describes what the page does without requiring prior knowledge of BLADE. BLADE can then be introduced as the built-in skill that powers it, which also naturally gives it the definition it needs. This leaves room for future skills without another rename.
adding all the changes asked by Seph and restarting the pr because of rebasing issues