Vals AI Finds No Router Beats Single Models on Cost and Accuracy
Summary
Vals AI's Router Index finds none of six model routers tested beats single models on both cost and accuracy. Nexus Auto with Sol (Fireworks) ranks first at 60.02% and $1.25 per test, but Vals reports DeepSeek V4.1 Flash 1017 scored higher for about a third of the cost. Vals tested 1,250 agentic tasks in finance, coding, law and tax.
Key Points
- Nexus Auto with Sol (Fireworks) trails GPT-6.1 Sol on Vibe Code Bench, 74.7% against 88.9%, Vals reports.
- Fugu Max (Sakana) takes a median of about 110 steps on Terminal-Bench, yet 45 of 66 tasks fail in all three runs, per Vals.
- Six of the eight benchmarks are private and run under zero-data-retention agreements, Vals states.