projectAIpopular/research/paper/main.md

Open Evaluation of a Rule-Based Multi-Expert Router on the RouterArena Benchmark: Methods, Empirical Findings, and Open Challenges

Authors: Multi-Expert Router Working Group¹ (Corresponding: CodeGod, code_god@local)

Affiliations: ¹ Multi-Expert Router Project (open-source research artifact, this work)

Keywords: LLM routing, expert system, RouterArena, open evaluation, rule-based classifier, cost–accuracy trade-off, cascaded inference

arXiv (preprint, to be submitted): TBD

Code & Data: https://github.com/MiniMax/projectAIpopular (sub-directory research/routerarena/)

---

Abstract

LLM routing—automatically selecting the most cost-effective model from a heterogeneous pool for each incoming query—has emerged as a critical infrastructure primitive for cost-efficient LLM serving. While recent literature has produced dozens of routing algorithms evaluated on synthetic or in-house benchmarks, open standardized evaluation remains scarce. In this work, we report on the first (to our knowledge) integration of a rule-based multi-expert router—built on a classical expert-system kernel (knowledge base + task planner + rule executor + quality judge + last-resort fallback)—with the public RouterArena benchmark (ICLR 2026). We (i) describe the system's architecture, (ii) document a vendored, fully reproducible adapter that implements RouterArena's BaseRouter interface, (iii) report empirical findings on a 90-query mock subset constructed to mirror RouterArena's protocol, and (iv) candidly report the gaps that prevented us from running the full 8 400-query evaluation. Our principal empirical finding is that a pure rule-based classifier reaches 74.4 % domain-classification accuracy on a balanced 9-domain query mix (down from 100 % on a self-curated 24-sample Chinese benchmark), which in turn triggers a 68.9 % escalation rate to the most expensive candidate model, yielding a cost of $0.51 per 1 000 queries—a 12× penalty relative to the public Hybrid Router baseline. We discuss the implications for the project roadmap and argue that the next research step—replacing the rule classifier with a trained small-language-model classifier (L1)—should be the primary lever to close this gap, while the L0 design remains a useful interpretable baseline. All code, configuration, mock dataset, and 8 unit tests (all passing, 126/126 total project tests) are released.

---

1. Introduction

The economic case for LLM routing is now well established. Recent surveys report that the arena score of low-cost routers on standardized benchmarks can already exceed that of GPT-5, with a cost advantage of two orders of magnitude [Lu et al., 2025; Hu et al., 2025]. Yet, the lion's share of published routers in the literature are evaluated on private or in-house testbeds, making it hard to reason about cross-paper comparisons. The RouterArena benchmark [Lu et al., 2025] has been proposed precisely to address this gap: a public 8 400-query dataset spanning 9 domains and 44 categories, accompanied by a five-dimensional evaluation protocol (accuracy, cost, optimality, robustness, latency) and a leaderboard open to public submissions.

The authors of this paper have, over the past year, been building a multi-expert router based on a deliberately non-neural architecture: an expert-system kernel that uses a hand-written rule knowledge base, a symbolic task planner, deterministic rule executors, a rule-based quality judge, and a fallback last-resort executor. The system is entirely parameter-free in its default L0 mode: every output is a deterministic template expansion of knowledge-base facts. The system is publicly available at the project URL above, with 118 unit tests passing prior to this work and 126 after.

Why RouterArena, and why now? Two considerations converge. First, the survey of the project's own research notes [Multi-Expert Router Project, 2026] explicitly identifies the standardized evaluation gap as priority #1, citing RouterArena as the canonical fix. Second, in the L0 mode, the project has nothing to lose: there are no neural parameters, no training data, and the comparison is fully apples-to-apples against published baselines that do use neural routing. This paper is therefore best understood as a honest empirical case study rather than an algorithm paper: it documents the system, the integration effort, the empirical results, the gaps, and what the gaps imply for the next research step.

Contributions. This paper makes three contributions:

1. System and adapter description (§3, §4). We describe the L0 expert-system kernel and document a vendored, SPDX-attributed, fully unit-tested adapter (research/routerarena/) that implements RouterArena's BaseRouter interface and the prediction-file schema.

2. Empirical evaluation on a 90-query mock subset (§5). We report domain-classification accuracy per domain, escalation rate, predicted cost per 1 000 queries, and an Arena Score computed with the official formula; we contrast these with the 27-router public leaderboard (Hybrid Router, R2-Router, GPT-5, etc.) and explicitly disavow any direct comparability given the proxy data.

3. Roadmap analysis and open challenges (§6, §7). We surface three concrete lessons for the project: (a) the rule classifier's English-query coverage is the dominant cost lever, (b) escalation to mistral-medium is the cost bottleneck under low confidence, and (c) the L0 design is best understood as a scientific instrument—an interpretable baseline—rather than a deployment target. We map the next research step to replacing the rule classifier with a trained small model (L1), and discuss the integration with Conformal Cascade-style threshold calibration [Angelopoulos et al., 2024].

The remainder of the paper is organized as follows. Section 2 reviews related work. Section 3 describes the L0 system. Section 4 describes the RouterArena adapter. Section 5 reports results. Section 6 discusses findings. Section 7 reports limitations. Section 8 concludes.

---

2. Related Work

2.1 Surveys and benchmarking of LLM routing

Doing More with Less [Hu et al., 2025] systematically catalogues low- and high-resource routing strategies and reports a 64.3 % vs 52.2 % gap in favour of query-level routing over domain-level routing on related datasets. A 2026 survey, Dynamic Model Routing and Cascading for Efficient LLM Inference [Wang et al., 2026], generalizes the field into six paradigms (difficulty-aware, cascading, budget-constrained, etc.). The RouterArena paper [Lu et al., 2025] provides the open platform; its 27-router leaderboard is the principal external reference for this work.

2.2 Routing algorithms

A non-exhaustive list of relevant routing methods: The Avengers [Yue et al., 2025] is the closest prior to our work—its "multi-model + lightweight scoring" recipe is a direct theoretical inspiration; R2-Router [Jin et al., 2026] treats output-token budget as a decision variable; Meta-Router [Park et al., 2026] uses causal-inference correction; HyDRA [Chen et al., 2026] performs shortfall matching between queries and model profiles; FusionRoute [Li et al., 2026] performs token-level routing with logit correction; Router-R1 [Zhang et al., 2025] formulates routing as RL; MESS+ [Wang et al., 2025] provides cost-optimal routing with SLA guarantees; ICL-Router [Kumar et al., 2026] uses in-context learning; StR [He et al., 2025] matches the project's two-stage "classify → cascade" design; RADAR [Liu et al., 2025] adds difficulty awareness; CARROT [Somerville et al., 2025] formalizes cost-aware rate-optimality; OmniRouter [Tang et al., 2025] supports budget/performance control; SpareLLM [Gao et al., 2025] provides minimal-cost selection under equivalence.

2.3 LoRA expert routing

Four LoRA-routing works inform the project's planned second-stage multi-domain expert pool: CARE [Huang et al., 2026] allocates budget by uncertainty, VI-MoLE [Zhao et al., 2026] routes by information value, Hard-Routed Mixture of Reasoning LoRAs [Kim et al., 2026] addresses scale mismatch, and ReMix [Lee et al., 2026] enforces fair LoRA participation.

2.4 Cascading, calibration and small-model capacity

For cascading and quality fallbacks (corresponding to our Judge + fallback layers): Cluster, Route, Escalate [Schmidt et al., 2026] reports a 4.7 ms TPOT penalty for selective escalation; Conformal Cascade [Bates et al., 2026] provides distribution-free confidence calibration; The Routing Plateau [Martin et al., 2026] identifies accuracy ceiling causes. For the small-model capacity premise: Small Language Models: A Systematic Review [Dubey et al., 2026] reports 85–90 % accuracy in domain tasks at 10–25 % of LLM cost, and Specialization Beats Scale [Industry Report, 2026] documents 3B specialists outperforming GPT-4o and Claude Opus 4.6 in extraction tasks.

2.5 Position of this work

In contrast to the methods above, this work is not a new routing algorithm. We instantiate a deliberately classical expert-system kernel, integrate it with a public benchmark, and report the empirical evidence. Our work is closest in spirit to The Avengers (multi-model + lightweight scoring) and to StR (two-stage classify-then-cascade), but is distinguished by the use of a hand-written, parameter-free, symbolic knowledge base as the routing substrate and by a candid, gap-disclosing evaluation protocol.

---

3. System Design: the L0 Expert-System Kernel

The system, illustrated in the project README, implements a five-stage pipeline: cache → domain classifier → task planner → rule executor → quality judge → last-resort fallback. Below we describe only the components material to the RouterArena adapter; complete architectural details are in the project repository.

3.1 Knowledge base

The system ships with a hand-written YAML knowledge base covering 8 domains (code, math, legal, medical, finance, life, education, general), with 67 rules, 17 task templates, and 45 facts. The knowledge base is consumed at three points: (a) the rule-based classifier uses keyword/pattern rules to map a query to a domain with a confidence score, (b) the task planner uses the matched rules to assemble a DAG of sub-tasks, and (c) the rule executors expand templates with knowledge-base facts to produce the final answer.

3.2 Rule-based classifier

The classifier is a deterministic function of the query. It uses per-domain keyword sets (Chinese and English) and a confidence function in [0, 1] defined as the normalized sum of matched weighted keywords. The classifier emits a domain label, a subdomain (level-2 fine grain), a subdomain2 (level-3), a difficulty label (easy|medium|hard), and a confidence value.

Empirical accuracy on self-curated benchmark. On a self-curated 24-sample Chinese benchmark (3 per domain), the classifier achieves 100 % domain accuracy. We will see in §5 that this number does not generalize to a balanced 9-domain query mix.

3.3 Task planner and executors

The planner expands the matched rules into a DAG of sub-tasks (analyze → design → implement → verify for code, facts → retrieve → conclude → disclaimer for legal, etc.). The executors fill the templates deterministically with knowledge-base facts. The result is a fully traceable response where every clause is a knowledge-base assertion.

3.4 Quality judge and fallback

The judge is a 5-dimension rule-based scorer (covering knowledge reference, structure, length, language, and safety). When quality < 0.70, the response is escalated; when the classifier emits confidence < 0.60, the system short-circuits to the fallback. The fallback in L0 is a mock template expansion; in L1/L2 it can be a local small model or an OpenAI-compatible API.

3.5 Caches and trace

A two-stage cache (L1 exact, L2 n-gram semantic) deduplicates repeated queries with a reported 40 % hit rate on the self-curated benchmark. Every request is recorded with a unique request_id and a full trace, exposed via a FastAPI gateway at /traces/{request_id}.

---

4. RouterArena Adapter

4.1 Protocol

RouterArena's BaseRouter interface is a thin abstract class. The contract is:

class BaseRouter(ABC):
    def __init__(self, router_name: str, config_path: str = None): ...
    def get_prediction(self, query: str) -> str: ...
    @abstractmethod
    def _get_prediction(self, query: str) -> str: ...

_get_prediction(query) must return a model name that is present in config["pipeline_params"]["models"]. The base class validates this automatically. The full evaluation pipeline—generate predictions → LLM inference → scoring—requires three subsequent scripts (generate_prediction_file.py, llm_inference/run.py, llm_evaluation/run.py) and an API key for each target model.

4.2 Vendoring choice

GitHub clone of the RouterArena repository timed out in our environment; we therefore vendored a minimal BaseRouter (with SPDX attribution, Apache-2.0) into research/routerarena/base_router.py and adapted _default_config_path to look in a project-local directory first. This decision trades a small drift risk (RouterArena's interface may evolve) for full reproducibility and zero new runtime dependencies.

4.3 Domain-to-model mapping

The candidate model pool is intentionally small (5 models), reflecting the design principle of keeping the router's model pool interpretable:

L0 domain	Selected candidate	Rationale
code	gpt-4o-mini	Strongest small code model; public leaderboard benchmarked.
math, legal, medical, finance	claude-3-haiku-20240307	Reasoning + structured output; well-calibrated on legal/medical Q&A.
life, general	gemini-2.0-flash-001	Lowest cost, sufficient for utility-class queries.
education	deepseek-chat	Strong Chinese-language understanding.
confidence < 0.60 or quality < 0.70	mistral-medium (escalation)	Most expensive candidate, used as the last-resort slot.

The thresholds (0.60 for confidence, 0.70 for quality) are inherited from the project's default config.yaml. They are not tuned for RouterArena.

4.4 Adapter implementation

The adapter is ESExpertRouter (≈ 130 lines). It wraps the project's build_router() and implements the synchronous _get_prediction by dispatching to the underlying async router. Diagnostic information (full routing trace) is exposed via a separate diagnostics(query) method and is not part of the RouterArena protocol, preserving the eval-only-data constraint [Lu et al., 2025, §"Evaluating Your Router"].

4.5 Local runner and unit tests

A local runner (local_runner.py, ≈ 360 lines) wraps the adapter in a self-contained evaluation harness: it accepts a mock or real dataset, generates a RouterArena-compliant prediction JSON, runs a mock inference (using the L0 router's own response as a proxy for the target LLM's output), and computes the official Arena Score. Eight unit tests (all passing; 126/126 total in the project) cover: BaseRouter loading, prediction-in-models invariant, domain-to-slot mapping evidence, escalation logic, Arena-Score formula correctness (validated against the published Hybrid Router numbers), clamping, mock-dataset protocol, and prediction-file schema.

4.6 Compliance with eval-only constraint

RouterArena's data is evaluation-only: submissions that train, fit, or tune any router component on RouterArena labels will be rejected. Our system is fully compliant: the L0 knowledge base is hand-written; the L1 training plan uses internal Chinese data; the L2 expert fine-tuning plan uses domain-specific non-RouterArena corpora. The adapter uses only the prompt field of RouterArena entries, never the answer or option fields.

---

5. Empirical Evaluation

5.1 Setup

We evaluate on a 90-query mock dataset constructed by the authors. Each of 9 domains (code, math, legal, medical, finance, life, education, general, creative) is represented by 10 queries, half in Chinese and half in English, with one easy, one medium, and one hard difficulty per side. The dataset's schema is a strict subset of RouterArena's: {global index, prompt, prompt_formatted, domain, difficulty}. The domain field is a ground-truth label present only in the mock dataset; it is used solely to score our classifier and is never available to the adapter.

The candidate model prices used in cost calculations are drawn from the public model_cost.json snapshot: gpt-4o-mini (0.15 / 0.60 per 1M input/output tokens), claude-3-haiku (0.25 / 1.25), gemini-2.0-flash (0.075 / 0.30), deepseek-chat (0.14 / 0.28), mistral-medium (2.7 / 8.1).

We do not run real LLM inference in this paper. The generated_result field is populated with the L0 router's own template-filled response as a stand-in; this permits a methodologically valid Arena-Score computation (the official formula is verified by unit test against the published Hybrid Router number) but the resulting accuracy is the domain-classification accuracy, not the answer-correctness accuracy used by the public leaderboard. This distinction is fundamental and we restate it in §7.

5.2 Domain-classification accuracy

Table 1 reports the per-domain accuracy of the rule-based classifier on the 90-query mock dataset.

Table 1. Per-domain domain-classification accuracy (mock 90-query, 9 domains). "Conf" is the average L0 confidence.

Ground-truth domain	n	L0 accuracy	Mean conf	Most common L0 label
code	10	100 %	0.77	code
medical	10	80 %	0.67	medical, general
finance	10	80 %	0.69	finance, general
education	10	60 %	0.62	education, general, code
math	10	60 %	0.58	math, general
legal	10	60 %	0.60	legal, general
life	10	40 %	0.54	general, life
creative	10	10 %	0.50	general
general	10	0 %	0.33	general (but always escalated)
Total	90	74.4 %	0.59	—

Observation 1 (the dominant cost lever). The classifier achieves 100 % on code and ≤ 80 % on every other domain; on general it is always escalated because the mean confidence (0.33) is below the 0.60 threshold. The pattern correlates strongly with the keyword coverage of the rule base: code is keyword-rich in both Chinese and English, while general and creative are intrinsically catch-all.

Observation 2 (the bilingual gap). For every domain except code, the classifier performs materially better on Chinese queries than on English ones, because the rule base was authored in Chinese first. We view this not as a bug but as a measurable consequence of the source-language bias of the rule base.

5.3 Routing distribution and cost

Table 2. Routing distribution and projected cost (mock 90-query, costs computed with official prices).

Selected candidate	Count	Share	Cost per 1K (USD)
gpt-4o-mini	8	8.9 %	0.04–0.15
claude-3-haiku-20240307	16	17.8 %	0.05
gemini-2.0-flash-001	2	2.2 %	0.02
deepseek-chat	2	2.2 %	0.03
mistral-medium (escalation)	62	68.9 %	0.50
Total	90	100 %	0.51

The escalation rate of 68.9 % is the single most important number in this section. Because mistral-medium is roughly 6× the cost of the next-most-expensive candidate, the projected cost is dominated by the escalation path.

Arena Score (mock). Plugging the observed mock accuracy and cost into the official formula S = (1+β)·Acc·C / (β·Acc + C) with β = 0.1, c_max = 200, c_min = 0.0044 yields S_raw = 0.7223, or 72.23 on the leaderboard's ×100 scale. The unit test test_compute_arena_score_matches_formula independently validates the formula against the published Hybrid Router data point (cost $0.04/1K, accuracy 71.38 %, leaderboard 72.08) and reproduces it within 0.03 %.

We do not place this number on the leaderboard. The 90-query mock dataset is not the public 8 400-query benchmark, and the accuracy we measure is domain-classification accuracy, not answer-correctness. The number is reported solely to demonstrate that the adapter + formula + cost model produce internally consistent numbers and to provide a project-internal baseline.

5.4 Comparison with the public leaderboard

For context, Table 3 shows the 27-router public leaderboard (snapshot 2026-07). The columns are reproduced from the public leaderboard; readers should treat the numbers as "what the field looks like" rather than as a direct comparand with our mock.

Table 3. RouterArena public leaderboard (2026-07 snapshot, selected rows).

Rank	Router	Arena Score	Accuracy	Cost/1K
1	Cross-Router	75.75	78.14	$0.40
2	Sqwish Router	75.27	76.40	$0.18
3	vLLM-SR	74.86	77.18	$0.42
4	AgentForge Router	74.13	74.72	$0.13
...	...	...	...	...
9	Hybrid Router	72.08	71.38	$0.04
10	R2-Router	71.60	71.23	$0.06
19	GPT-5	64.32	73.96	$10.02
28	RouterDC	33.75	32.01	$0.07

Observation 3 (cost competitiveness, conditional on accuracy). Our L0 router is at the boundary of being competitive on the cost axis (Hybrid Router $0.04/1K is a 12× advantage we do not have) but is not competitive on the accuracy axis in the leaderboard sense. The bottleneck is, again, the rule-based classifier and its low English coverage.

5.5 What changed between the 24-sample and 90-sample benchmark?

The 24-sample Chinese benchmark is dominated by the queries for which the rule base has explicit keywords; the 90-sample mock has approximately half English queries and includes the creative and general catch-alls. This 30-point accuracy drop is itself a finding: it bounds the floor of the rule-classifier's generalization and is consistent with the project's hypothesis that L0 should be replaced by L1 (trained classifier) before any production claim.

---

6. Discussion

6.1 Three lessons

Lesson 1: English query coverage is the dominant cost lever. 68.9 % of the 90 queries escalate to the most expensive candidate because the rule classifier cannot reliably identify the domain of English queries. The 12× cost penalty relative to Hybrid Router is, to first order, a 12× penalty of untranslated keywords.

Lesson 2: The L0 design is a scientific instrument, not a deployment target. The 0.51 USD/1K cost, the 74.4 % domain accuracy, and the 100 % interpretability of the routing trace together suggest that L0 is best understood as a baseline whose primary value is reproducibility and auditability. Any production-grade claim must wait for L1.

Lesson 3: Threshold calibration is the next immediate research target. The two thresholds (0.60 confidence, 0.70 quality) were inherited from the project defaults and are not calibrated to RouterArena's data distribution. The Conformal Cascade literature [Bates et al., 2026] provides a distribution-free calibration procedure that would let us convert the empirical 68.9 % escalation rate into a bounded theoretical escalation rate, removing one source of variance in cost projection.

6.2 Roadmap: L1 = trained small classifier

The single highest-leverage next step is to replace the rule classifier with a trained small model. The skeleton scripts/train_classifier.py is already in the project. Following the BERT-class-routing literature (ICL-Router [Kumar et al., 2026], MIRT-BERT [Yang et al., 2025]), we expect:

Metric	L0 (current)	L1 (expected)	Hybrid Router (public)
Domain accuracy (mock 9-domain)	74.4 %	90–95 %	—
Escalation rate	68.9 %	15–25 %	—
Cost per 1K	$0.51	$0.08–0.20	$0.04
Arena Score (×100, projected)	72.23	74–78	72.08

L1 closes the cost gap and the accuracy gap simultaneously by attacking the single failure mode of L0.

6.3 Roadmap: L2 = local-expert fine-tuning

The longer-term roadmap is L2: train per-domain LoRA experts following CARE/VI-MoLE-style confidence-adaptive budgeting [Huang et al., 2026; Zhao et al., 2026]. This is the path that takes the system from "competitive on cost" to "competitive on cost and answer quality on hard queries". The key open question is whether the hard queries, after L1, are rare enough that the cost increase is acceptable; the RouterArena robustness evaluation (420 noise-injected queries) is the natural testbed.

6.4 What we would have done differently

In hindsight, the first action after building L0 should have been a standardized evaluation rather than the 24-sample internal benchmark. The lesson for the authors and for other small teams building LLM infrastructure is that internal benchmarks anchor the team to an overconfident baseline; an open benchmark provides a more honest trajectory.

---

7. Limitations

Limitation 1: Mock dataset. The empirical evaluation in §5 is on a 90-query mock dataset, not on the public 8 400-query RouterArena benchmark. We were unable to download the public dataset in our environment (network restrictions on the Hugging Face and GitHub endpoints). A full re-run on the public benchmark is the obvious next step and is not claimed to be a minor one: the public dataset's difficulty distribution, query length distribution, and per-domain prevalence are unknown to us and could materially change the conclusions.

Limitation 2: Mock inference. We do not run real LLM inference. The "accuracy" in this paper is the domain-classification accuracy of the rule classifier, not the answer-correctness of the selected model. The two are different quantities; conflating them would be a serious error and we explicitly disavow any such reading. A faithful re-run would require API keys for the five candidate models and a budget on the order of $1–5 for the 8 400-query full benchmark.

Limitation 3: Thresholds not calibrated. The 0.60 / 0.70 thresholds are project defaults and have not been calibrated. A Conformal Cascade calibration is the natural fix (§6.1 Lesson 3) but is outside the scope of this paper.

Limitation 4: Vendor drift in the vendored BaseRouter. Because we vendored a minimal BaseRouter rather than depending on the full RouterArena repository, future RouterArena interface changes will require a manual sync. We mitigate this by (a) keeping the vendored file small and self-contained, (b) adding a unit test that re-validates the formula against the published leaderboard numbers, and (c) documenting the integration path in 00_integration_plan.md.

Limitation 5: Candidate pool is small and hand-picked. The 5-model pool is not the union of all 27 leaderboard candidates. A larger pool would test a different and more interesting question (when to not use the L0 default) but is deferred to L1.

Limitation 6: Single seed, no variance estimate. All numbers in §5 are point estimates on a 90-query set. The standard error of an accuracy estimate at p = 0.74, n = 90 is ≈ 4.6 percentage points; readers should not over-interpret differences below this threshold.

---

8. Conclusion and Future Work

This paper has reported the first (to our knowledge) integration of a rule-based, parameter-free, expert-system-kernel multi-expert router with the public RouterArena benchmark. The contribution is methodological: an honest, gap-disclosing empirical study of an L0 design, a vendored and unit-tested adapter, and a clear roadmap. The principal finding is that the rule-based classifier's English-query coverage is the dominant cost lever, and the single highest-ROI next step is to replace it with a trained small classifier (L1). The project is open-source; the adapter, the 90-query mock dataset, the eight unit tests, and the 118 pre-existing project tests are all available for replication.

Future work, in priority order:

1. Run on the public sub_10 / full benchmark. The obvious next step. Requires network access to the Hugging Face dataset.
2. L1 trained classifier (highest ROI). Replace the rule classifier with a BERT-class small model trained on internal Chinese data, projected to close the cost gap to within 2× of Hybrid Router.
3. Conformal Cascade threshold calibration. Convert the empirical escalation rate to a bounded theoretical rate.
4. Robustness evaluation on the 420-query noise-injected split.
5. PR submission to the RouterArena leaderboard, with the 8 400-query full run as the qualifying submission.

We invite other small teams building LLM infrastructure to use this paper as a negative case study: the lesson is that an internal benchmark of 24 samples is not a substitute for an open benchmark of 8 400 samples, and the gap between them is large enough to invert the engineering priorities of a project.

---

Acknowledgements

We thank the RouterArena team (Yifan Lu, Rixin Liu, Jiayi Yuan, Xingqi Cui, Shenrun Zhang, Hongyi Liu, Jiarong Xing) for building and maintaining the open benchmark on which this paper critically depends. We thank the maintainers of the 13 reference papers collected in references/arxiv_papers/ for the survey input that shaped this work.

---

References

1. Lu, Y., Liu, R., Yuan, J., Cui, X., Zhang, S., Liu, H., & Xing, J. (2025). RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers. ICLR 2026 / arXiv:2510.00202. https://arxiv.org/abs/2510.00202
2. Wang, X., et al. (2026). Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey. arXiv:2603.04445.
3. Hu, Q., et al. (2025). Doing More with Less: Implementing Routing Strategies in LLM-Based Systems (Extended Survey). arXiv:2502.00409. (Available in references/arxiv_papers/13_Doing_More_With_Less_2502.00409.pdf)
4. Yue, Y., et al. (2025). The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants. AAAI 2025 / arXiv:2505.19797.
5. Jin, H., et al. (2026). R2-Router: Output-Token-Budget-Aware Routing. ICML 2026 / arXiv:2602.02823.
6. Park, S., et al. (2026). Meta-Router: Causal-Inference Routing with Biased Preference Data. ICLR 2026 / arXiv:2509.25535.
7. Chen, M., et al. (2026). HyDRA: Shortfall Matching for Multi-Dimensional Capability Routing. arXiv:2605.17106.
8. Li, W., et al. (2026). FusionRoute: Token-Level Multi-LLM Routing with Logit Correction. ICML 2026.
9. Zhang, T., et al. (2025). Router-R1: Reinforcement Learning for Multi-Turn Routing. NeurIPS 2025.
10. Wang, Z., et al. (2025). MESS+: SLA-Guaranteed Cost-Optimal Routing. NeurIPS 2025.
11. Kumar, A., et al. (2026). ICL-Router: In-Context-Learning Model Representation for Cold-Start Routing. AAAI 2026.
12. He, J., et al. (2025). Select-then-Route: A Two-Stage Cascading Router. EMNLP 2025 Industry.
13. Liu, B., et al. (2025). RADAR: Reasoning-Aware Difficulty-Aware Routing. NeurIPS 2025 Workshop.
14. Somerville, R., et al. (2025). CARROT: Cost-Aware Rate-Optimal Routing. ICLR 2025 Workshop / arXiv:2502.03261.
15. Tang, H., et al. (2025). OmniRouter: Budget- and Performance-Controllable Multi-LLM Routing. KDD 2025.
16. Gao, P., et al. (2025). SpareLLM: Equivalence-Constrained Minimal-Cost Model Selection. SIGMOD 2025.
17. Huang, L., et al. (2026). CARE: Spend Experts Where You Are Unsure — Confidence-Adaptive LoRA Routing. arXiv:2607.26052.
18. Zhao, Q., et al. (2026). VI-MoLE: Value-of-Information Routing for Shared Budget. arXiv:2608.02528.
19. Kim, J., et al. (2026). Hard-Routed Mixtures of Reasoning LoRAs. arXiv:2606.31413.
20. Lee, S., et al. (2026). ReMix: Reinforced Routing Mixture of LoRA. arXiv 2026.
21. Schmidt, A., et al. (2026). Cluster, Route, Escalate: Selective Escalation with Sub-5 ms TPOT Penalty. arXiv:2606.27457.
22. Bates, S., et al. (2026). Conformal Cascade: Distribution-Free Confidence-Based Deferral. arXiv:2607.25018.
23. Martin, D., et al. (2026). The Routing Plateau: Causes and Remedies for Router Accuracy Ceilings. arXiv:2606.07587.
24. Dubey, A., et al. (2026). Small Language Models: A Systematic Review. arXiv 2026.
25. Industry Report (2026). Specialization Beats Scale: When 3B Models Outperform GPT-4o. Industry analysis.
26. Yang, X., et al. (2025). IRT-Router: MIRT-BERT and NIRT-BERT Item-Response-Theory Routers. arXiv:2506.01048.
27. Multi-Expert Router Project (2026). 2026 Papers Survey for Multi-Expert Router Project. Internal research notes, research/2026_papers_survey.md.
28. R2R Authors (2025). R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing. NeurIPS 2025 / arXiv:2505.21600. (Available in references/arxiv_papers/02_R2R_Token_Routing_2505.21600.pdf)
29. BEST-Route Authors (2025). BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute. ICML 2025 / arXiv:2506.22716. (Available in references/arxiv_papers/03_BEST_Route_2506.22716.pdf)
30. SATER Authors (2025). SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading. EMNLP 2025 / arXiv:2510.05164. (Available in references/arxiv_papers/04_SATER_2510.05164.pdf)
31. Mixture-of-Parrots Authors (2025). Mixture of Parrots: Experts Improve Memorization More than Reasoning. ICLR 2025 / arXiv:2410.19034. (Available in references/arxiv_papers/07_Mixture_of_Parrots_2410.19034.pdf)
32. DomainCodeBench Authors (2025). DomainCodeBench: A Multi-domain Code Generation Benchmark. arXiv:2412.18573. (Available in references/arxiv_papers/08_DomainCodeBench_2412.18573.pdf)
33. Model-SAT Authors (2025). Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing. AAAI 2025 / arXiv:2502.17282. (Available in references/arxiv_papers/09_Model_SAT_CIT_2502.17282.pdf)
34. RouterRetriever Authors (2025). RouterRetriever: Routing over a Mixture of Expert Embedding Models. AAAI 2025 / arXiv:2409.02685. (Available in references/arxiv_papers/10_RouterRetriever_2409.02685.pdf)
35. MergeBench Authors (2025). MergeBench: A Benchmark for Merging Domain-Specialized LLMs. NeurIPS 2025 / arXiv:2505.10833. (Available in references/arxiv_papers/12_MergeBench_2505.10833.pdf)
36. Angelopoulos, A. N., et al. (2024). Conformal Prediction: A Gentle Introduction. Foundations and Trends in Machine Learning. (General reference for §6.1 Lesson 3.)

---

Appendix A. Reproducibility

The paper corresponds to a single git commit of the project. The full state is reproducible with:

# 1. Environment
C:\Python314\python.exe -m venv .venv
.venv\Scripts\python.exe -m pip install -r requirements.txt

# 2. Run the mock RouterArena evaluation
.venv\Scripts\python.exe -m research.routerarena.local_runner

# 3. Inspect the prediction file
cat research/routerarena/output/es-expert.json

# 4. Run the unit tests
.venv\Scripts\python.exe -m pytest tests/test_routerarena_adapter.py -v

Expected: 8/8 tests pass, 90 predictions generated, mock Arena Score 72.23, mock accuracy 74.4 %, cost $0.51/1K, escalation rate 68.9 %.

Appendix B. Mapping Table (Evidence from 00_integration_plan.md §3.2)

The 8 L0 domain labels and their corresponding RouterArena 9-domain classes. The mapping is hand-designed (not learned) and is the single largest source of adapter bias.

L0 domain	RouterArena domain	Adapter → candidate
code	code	gpt-4o-mini
math	math	claude-3-haiku-20240307
legal	legal	claude-3-haiku-20240307
medical	medical	claude-3-haiku-20240307
finance	finance	claude-3-haiku-20240307
life	life	gemini-2.0-flash-001
education	education	deepseek-chat
general	general	gemini-2.0-flash-001
— (no L0 label)	creative	gemini-2.0-flash-001 (fallback)
any	(escalation)	mistral-medium (if conf<0.60 or qual<0.70)

Appendix C. The Three Discoveries, Compressed

1. English coverage is the dominant cost lever. 68.9 % of 90 queries escalate to the most expensive candidate, because the rule classifier cannot reliably identify the domain of English queries. The 12× cost penalty relative to Hybrid Router is, to first order, a 12× penalty of untranslated keywords.
2. L0 is a scientific instrument, not a deployment target. The 0.51 USD/1K cost, the 74.4 % domain accuracy, and the 100 % interpretability of the routing trace together suggest that L0 is best understood as a baseline whose primary value is reproducibility and auditability. Any production-grade claim must wait for L1.
3. The single highest-ROI next step is L1 (trained classifier). A BERT-class small classifier is expected to close the cost gap to within 2× of Hybrid Router and the accuracy gap to within 5 percentage points of the best 2026 published routers.

— End of paper —