Muse Spark Open (Meta open weights) model evaluation
C · Weak signal A Vals AI benchmark listing page for Meta's open-weight model; independent scores, but no method write-up on the page and no Meta primary source for the model's specifications.
Summary based on the full text·Original: about 3 minutes to read
In short
Vals AI's model page for Muse Spark Open, an open-weight model from Meta listed as released 15 October 2026. It scores 46.61% on the Vals Index, 38th of 49 models, with a strong finance-agent result and weak coding and terminal results. The page gives context window, token costs and test settings but not parameter counts or licence terms.
Key numbers
- 46.61%Vals Index accuracy, rank 38 of 49 models
- 61.34%Finance Agent (v2) accuracy, rank 3 of 79
- 6.57%Terminal-Bench 4.0 accuracy, rank 40 of 49
- $1.25/4.25token costs in/out listed by Vals
As reported by the source.
English·Web page·opens vals.ai
What it reports
- Vals Index 46.61% (±1.20), rank 38/49; cost per test $4.077; latency 67 min 40 s.
- Best result: Finance Agent (v2) 61.34%, rank 3/79. Harvey's Legal Agent Benchmark 16.67%, rank 9/80.
- Weak spots: Code Migration 21.17% (51/78), Terminal-Bench 4.0 6.57% (40/49), Tax Agent Bench 26.32% (34/70).
- Context window 1M, max output 131k, text and image input; served via Together AI at temperature 1, reasoning effort xhigh.
How they got it
Vals AI runs its own and partner benchmarks through a provider endpoint. The page reports accuracy with error bars and rankings; benchmark methods sit on separate pages not read here.
Limits
- No Meta announcement was found; parameter counts and licence terms circulating elsewhere are unconfirmed.
- One provider and one setting; scores may differ under other configurations.
- A listing page, not an analysis.
Why it matters for companies
For firms weighing open weights for internal deployment, the page shows the usual pattern: competitive on a narrow domain task, well behind closed models on general and agentic coding work. Check the licence before any commercial use, and test on your own tasks rather than relying on a single index rank.
Summary based on the full text. Read the original for the full argument and data.Read the original on vals.ai