feat(composite): align defaults with unified routing design

benchmark/DEEPSWE_V11_COMPOSITE_REPORT.md

@@ -0,0 +1,68 @@
+# DeepSWE v1.1 Composite results
+
+Three full 113-task repeats measured Composite with GPT-6 Astra at medium effort,
+GPT-6 Luna at max effort, and Luna as the Decision model. The campaign ran from
+October 31 to November 1, 2026, using Switchyard revision
+`bdd7647b5d55be46b19764136128646edb08692e`, Codex CLI 0.154.0, and Harbor 0.13.2.
+They used closed-book execution and a maximum of five concurrent tasks.
+These historical results support the routing recipe. They do not measure the
+current PR commit.
+
+## Results
+
+Costs use recorded tokens at fixed base rates. Each cost is for a full 113-task run.
+
+| Configuration | Passes by repeat | Task completion | Mean cost per run |
+| --- | --- | ---: | ---: |
+| Composite CF0/EF0.4 | 79, 79, 82 | 240/339, 70.80% | $269.99 |
+| Astra only | 83, 78, 82 | 243/339, 71.68% | $362.19 |
+
+Composite used 25.45% less recorded-token cost, with task completion 0.88 percentage
+points lower. All tasks remain in the denominator, including two verifier timeouts
+in the second Composite repeat. Three repeats do not establish statistical
+equivalence or generalization to other workloads.
+
+The [JSON results](deepswe-v11-composite-results.json) include all six run IDs,
+pass counts, token totals, pricing rates, and source hashes.
+
+## Configuration
+
+The [route configuration](routing-profiles/deepswe-v11-composite-astra-luna.toml)
+expresses the measured settings using the current configuration schema. Merge it
+with deployment bindings for `astra` at medium effort, `luna` at max effort, and
+the `classifier` Decision target using Luna. Serving endpoints are deployment-specific.
+
+- One profile selection per session, with `classify_trigger = "new_session"`.
+- Decision cutoff `0.92`, with capable-first selected only above the cutoff.
+- Efficient-first threshold `0.4`, capable hold `2`, efficient hold `0`.
+- Capable-first threshold `0.0`, capable hold `0`, efficient hold `-1`.
+- A window of three recent tool results and the capable-first planning prompt.
+- No inner judge or handoff notes. The measured configuration used `fail_open = false`.
+
+The current default [Decision instructions](https://github.com/NVIDIA-NeMo/Switchyard/blob/5bde8855556c242d4a994b5d985a2d66947f7665/crates/libsy/src/algorithms/llm_class/decision.rs)
+and [Composite planning prompt](https://github.com/NVIDIA-NeMo/Switchyard/blob/5bde8855556c242d4a994b5d985a2d66947f7665/crates/switchyard-runner/src/algorithm.rs)
+match the saved recipe, with choice IDs renamed from `capable_first` and
+`efficient_first` to `capable` and `efficient`. The route file inherits those
+prompts. Interactive `user_turn` selection was not measured in these runs.
+
+## Cost accounting and coverage
+
+The same rates apply to both configurations, in USD per million tokens:
+
+| Model | Input | Cached input | Cache creation | Output |
+| --- | ---: | ---: | ---: | ---: |
+| GPT-6 Astra | 10.00 | 1.00 | 12.50 | 50.00 |
+| GPT-6 Luna | 0.10 | 0.01 | 0.125 | 0.50 |
+
+Prompt-token totals include cached input and cache creation. Subtract both before
+pricing ordinary input. Recorded selector tokens are included. Infrastructure and
+verifier costs are outside this model-token comparison.
+
+Two interrupted Composite answer calls have missing usage. The Astra baseline has
+priced usage for 109, 111, and 111 tasks. Its shared logs also contain unmatched
+requests that cannot all be attributed to these runs. Per-request long-context
+pricing is unverified. The 25.45% reduction compares incomplete recorded-token
+estimates, not complete bills or a guaranteed savings bound.
+
+Tuning and evaluation task overlap has not been independently verified. The live
+results above should not be presented as held-out validation.

benchmark/deepswe-v11-composite-results.json

@@ -0,0 +1,208 @@
+{
+  "benchmark": "DeepSWE v1.1",
+  "benchmark_revision": 1,
+  "tasks_per_repeat": 113,
+  "scoring": "Reward 1 counts as a pass. All 113 tasks remain in every denominator.",
+  "cost_basis": "Recorded model tokens at fixed base rates. Incomplete usage and per-request context tiers prevent a complete billed-cost claim.",
+  "rates_usd_per_million_tokens": {
+    "gpt-6-astra": {
+      "input": 10,
+      "cached_input": 1,
+      "cache_creation": 12.5,
+      "output": 50
+    },
+    "gpt-6-luna": {
+      "input": 0.1,
+      "cached_input": 0.01,
+      "cache_creation": 0.125,
+      "output": 0.5
+    }
+  },
+  "campaigns": [
+    {
+      "name": "Composite CF0/EF0.4",
+      "repeats": 3,
+      "passed": 240,
+      "attempts": 339,
+      "accuracy_percent": 70.79646017699115,
+      "mean_recorded_token_cost_usd": 269.994194735,
+      "runs": [
+        {
+          "run_id": "bmr_d04a102d53194beaae0b38c323",
+          "tasks": 113,
+          "passed": 79,
+          "recorded_token_cost_usd": 276.633197045,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 1992,
+              "prompt_tokens": 105554383,
+              "completion_tokens": 1734883,
+              "cached_tokens": 98884683,
+              "cache_creation_tokens": 6663724
+            },
+            "gpt-6-luna": {
+              "calls": 4825,
+              "prompt_tokens": 409398780,
+              "completion_tokens": 3968251,
+              "cached_tokens": 395724617,
+              "cache_creation_tokens": 13570643
+            }
+          },
+          "tasks_with_priced_usage": 113,
+          "selector_calls_included": 113,
+          "answer_calls_missing_usage": 2,
+          "initial_profile_split": {
+            "CF": 55,
+            "EF": 58
+          },
+          "runtime_errors": 0,
+          "recovered_stream_disconnects": 2
+        },
+        {
+          "run_id": "bmr_758ff230283840288fcd852460",
+          "tasks": 113,
+          "passed": 79,
+          "recorded_token_cost_usd": 236.390371345,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 1783,
+              "prompt_tokens": 82655166,
+              "completion_tokens": 1522243,
+              "cached_tokens": 76577516,
+              "cache_creation_tokens": 6072301
+            },
+            "gpt-6-luna": {
+              "calls": 4853,
+              "prompt_tokens": 417582002,
+              "completion_tokens": 4037439,
+              "cached_tokens": 404090897,
+              "cache_creation_tokens": 13388555
+            }
+          },
+          "tasks_with_priced_usage": 113,
+          "selector_calls_included": 113,
+          "answer_calls_missing_usage": 0,
+          "initial_profile_split": {
+            "EF": 52,
+            "CF": 61
+          },
+          "runtime_errors": 2,
+          "recovered_stream_disconnects": 0
+        },
+        {
+          "run_id": "bmr_31cafdd1d20941839e87d953cd",
+          "tasks": 113,
+          "passed": 82,
+          "recorded_token_cost_usd": 296.959015815,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 2082,
+              "prompt_tokens": 119407906,
+              "completion_tokens": 1822748,
+              "cached_tokens": 112515394,
+              "cache_creation_tokens": 6886266
+            },
+            "gpt-6-luna": {
+              "calls": 4475,
+              "prompt_tokens": 377823990,
+              "completion_tokens": 3790902,
+              "cached_tokens": 364830234,
+              "cache_creation_tokens": 12892315
+            }
+          },
+          "tasks_with_priced_usage": 113,
+          "selector_calls_included": 113,
+          "answer_calls_missing_usage": 0,
+          "initial_profile_split": {
+            "EF": 59,
+            "CF": 54
+          },
+          "runtime_errors": 0,
+          "recovered_stream_disconnects": 0
+        }
+      ]
+    },
+    {
+      "name": "GPT-6 Astra baseline",
+      "repeats": 3,
+      "passed": 243,
+      "attempts": 339,
+      "accuracy_percent": 71.68141592920354,
+      "mean_recorded_token_cost_usd": 362.185674,
+      "runs": [
+        {
+          "run_id": "bmr_d2123a3199d34856992f9d8d3e",
+          "tasks": 113,
+          "passed": 83,
+          "recorded_token_cost_usd": 356.7353245,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 4071,
+              "prompt_tokens": 170935373,
+              "cached_tokens": 164502447,
+              "cache_creation_tokens": 6420167,
+              "completion_tokens": 2237064
+            }
+          },
+          "tasks_with_priced_usage": 109,
+          "unmatched_routing_requests": 214
+        },
+        {
+          "run_id": "bmr_fa6ac56b867b46f4b64fdadcc3",
+          "tasks": 113,
+          "passed": 78,
+          "recorded_token_cost_usd": 366.437225,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 4076,
+              "prompt_tokens": 171696488,
+              "cached_tokens": 164867070,
+              "cache_creation_tokens": 6817190,
+              "completion_tokens": 2324660
+            }
+          },
+          "tasks_with_priced_usage": 111,
+          "unmatched_routing_requests": 182
+        },
+        {
+          "run_id": "bmr_54841b35c74b46c0a1849b3f23",
+          "tasks": 113,
+          "passed": 82,
+          "recorded_token_cost_usd": 363.3844725,
+          "usage_by_model": {
+            "gpt-6-astra": {
+              "calls": 4040,
+              "prompt_tokens": 172015485,
+              "cached_tokens": 165225160,
+              "cache_creation_tokens": 6778205,
+              "completion_tokens": 2266211
+            }
+          },
+          "tasks_with_priced_usage": 111,
+          "unmatched_routing_requests": 199
+        }
+      ]
+    }
+  ],
+  "comparison": {
+    "accuracy_delta_percentage_points": -0.8849557522123916,
+    "recorded_token_cost_reduction_percent": 25.454203709062217
+  },
+  "provenance": {
+    "composite_run_dates_utc": [
+      "2026-10-31",
+      "2026-11-01"
+    ],
+    "composite_switchyard_commit": "bdd7647b5d55be46b19764136128646edb08692e",
+    "composite_routing_config_sha256": "025ddd73945162602e1d4db86934b5208223e7e3e6c0fbeef24d1252ea881264",
+    "composite_source_report_sha256": "5cf5e3411176ece5820beab37283b260484d1421cb1ce1f6ba06eedac69a2bb2",
+    "baseline_source_report_sha256": "e47fc2ce598078ea204b55bd6ae072c13ee67046f0407e46bfd319aeec7b7fbf"
+  },
+  "limitations": [
+    "Composite has two interrupted answer calls with missing usage. Recorded selector tokens are included.",
+    "Baseline has priced usage for 109, 111, and 111 tasks. Unmatched requests in shared logs cannot all be attributed to these runs.",
+    "The comparison uses base token rates, including cache creation at 1.25 times input. Per-request long-context pricing is unverified.",
+    "These are historical runs, not a benchmark of the current PR commit.",
+    "Tuning and evaluation task overlap has not been independently verified. Three repeats do not establish statistical equivalence or generalization."
+  ]
+}

benchmark/routing-profiles/deepswe-v11-composite-astra-luna.toml

@@ -0,0 +1,31 @@
+# Merge with deployment bindings for Astra medium, Luna max, and a Luna Decision target.
+# Uses the default Decision instructions and Composite planning prompt.
+
+[routes.switchyard]
+id = "switchyard"
+type = "composite"
+
+[routes.switchyard.classifier]
+target = "classifier"
+classify_trigger = "new_session"
+fail_open = false
+
+[routes.switchyard.classifier.decision]
+cutoff = 0.92
+
+[routes.switchyard.stage]
+capable_target = "astra"
+efficient_target = "luna"
+recent_turn_window = 3
+
+[routes.switchyard.efficient]
+picker = "efficient_first"
+confidence_threshold = 0.4
+capable_hold_turns = 2
+efficient_hold_turns = 0
+
+[routes.switchyard.capable]
+picker = "capable_first"
+confidence_threshold = 0.0
+capable_hold_turns = 0
+efficient_hold_turns = -1