A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost
B · Useful, limited A Hightouch engineering benchmark with full tables of accuracy, errors, latency and cost per model on a disclosed synthetic dataset; clear and checkable in setup, but a vendor blog on 1,000 synthetic identities per test.
Summary based on the full text·Original: about 10 minutes to read
In short
A 4 November 2026 Hightouch blog post by Albert Jin, Neil Desai and Delaney Mackenzie. They test whether cheap "decision models" that return a 0-1 confidence score (Jev and GPT-6 Luna) can grade identity-resolution output, specifically spotting profiles that wrongly merge several people. Alone they trail frontier models, more so on hard cases. Routing Jev's uncertain cases to Claude Opus 5.5 gets near-Opus accuracy at under a quarter of the cost, because Jev's confidence is well calibrated and Luna's is not.
Key numbers
- 99.0% at $3.86Claude Opus 5.5 accuracy and cost per 1,000 identities in grading identity-resolution results
- 98.8% at $0.88hybrid: Jev decides, sending 22% of cases in its 30-90% confidence band to Opus
- 95.7% at $0.05Jev alone at a 60% cutoff, answering in about 0.15 seconds
- 95.3% vs 95.1%hybrid Jev vs Opus alone on a harder test of tricky cases ($2.94 vs $5.39 per 1,000)
- 91.6%hybrid GPT-6 Luna on the harder test, held back by overconfident scores
As reported by the source.
English·Web page·opens hightouch.com
What it reports
- Benchmark: 1,000 identities built from pseudopeople synthetic records with added noise, half single people and half over-merges.
- Easier test, best prompt: Opus 5.5 99.0% ($3.86 per 1,000, 2.2 s), Gemini 3.1 Pro 98.3% ($4.48), GPT-6 Sol 98.3% ($2.50), GPT-6 Luna 97.3% ($0.10, 0.17 s), Jev 95.7% at a 60% cutoff ($0.05, 0.15 s).
- Every model gained most from listing example data errors in the prompt; source context added at most a point for frontier models.
- Calibration: most of Jev's mistakes sit near 50% confidence; Luna's mistakes cluster at 90-100% confidence.
- Hybrid Jev (Opus reviews the 30-90% band, 22% of cases): 98.8% at $0.88. Hybrid Luna (10-90% band): 98.5% at $0.33.
- Harder test (twins, juniors and seniors, same names, major typos): Opus 95.1% ($5.39), hybrid Jev 95.3% ($2.94), hybrid Luna 91.6% ($1.15), Jev alone 89.1%, Luna alone 88.8%.
- Human review runs at about 50 identities an hour; estimating a rare over-merge rate within plus or minus 1% needs more than 1,000 samples.
How they got it
Synthetic records from a simulated US town (University of Washington IHME's pseudopeople), assembled into labelled identities, graded by each model with the same prompt. Thresholds were tuned for accuracy, then cheap-model scores in a confidence band were forwarded to Opus. Costs and latencies are as observed in Hightouch's setup.
Limits
- Synthetic data only, 1,000 identities per test; no real customer graphs.
- Thresholds were tuned on the same data they are reported on.
- Measures over-merging only, not under-merging; written by a vendor that sells identity resolution, with a hiring call to action.
Why it matters for companies
A cheap model that knows when it is unsure, plus a frontier model for the uncertain cases, can match frontier accuracy at a fraction of the cost and spend more automatically when data get harder. The key selection criterion is calibration, not raw accuracy: Luna scored higher alone but was worse in the hybrid. Teams using LLMs to audit data quality, such as customer matching, should check where a model's errors fall on its confidence scale before designing the routing.
Summary based on the full text. Read the original for the full argument and data.Read the original on hightouch.com