---
id: "lib-2026-11-04-a-jev-that-can-ask-opus-for-help-gives-opus"
type: "library"
title: "A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost"
lang: "EN"
org: "Hightouch"
source_type: "Practitioner post"
date: "2026-11-04"
week: "2026-W45"
url: "https://nosov.ai/library/lib-2026-11-04-a-jev-that-can-ask-opus-for-help-gives-opus/"
original_url: "https://hightouch.com/blog/llm-judges-identity-resolution"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "A Hightouch engineering benchmark with full tables of accuracy, errors, latency and cost per model on a disclosed synthetic dataset; clear and checkable in setup, but a vendor blog on 1,000 synthetic identities per test."
summary_basis: "full-text"
original_minutes: "10"
tags: ["ai technology", "enablement", "llm as judge", "model routing", "inference cost"]
---

# A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost

Tier B (Useful, limited): A Hightouch engineering benchmark with full tables of accuracy, errors, latency and cost per model on a disclosed synthetic dataset; clear and checkable in setup, but a vendor blog on 1,000 synthetic identities per test.

Summary based on the full text.

## In short

A 4 November 2026 Hightouch blog post by Albert Jin, Neil Desai and Delaney Mackenzie. They test whether cheap "decision models" that return a 0-1 confidence score (Jev and GPT-6 Luna) can grade identity-resolution output, specifically spotting profiles that wrongly merge several people. Alone they trail frontier models, more so on hard cases. Routing Jev's uncertain cases to Claude Opus 5.5 gets near-Opus accuracy at under a quarter of the cost, because Jev's confidence is well calibrated and Luna's is not.

## Key numbers

- 99.0% at $3.86: Claude Opus 5.5 accuracy and cost per 1,000 identities in grading identity-resolution results
- 98.8% at $0.88: hybrid: Jev decides, sending 22% of cases in its 30-90% confidence band to Opus
- 95.7% at $0.05: Jev alone at a 60% cutoff, answering in about 0.15 seconds
- 95.3% vs 95.1%: hybrid Jev vs Opus alone on a harder test of tricky cases ($2.94 vs $5.39 per 1,000)
- 91.6%: hybrid GPT-6 Luna on the harder test, held back by overconfident scores

## What it reports

- Benchmark: 1,000 identities built from pseudopeople synthetic records with added noise, half single people and half over-merges.
- Easier test, best prompt: Opus 5.5 99.0% ($3.86 per 1,000, 2.2 s), Gemini 3.1 Pro 98.3% ($4.48), GPT-6 Sol 98.3% ($2.50), GPT-6 Luna 97.3% ($0.10, 0.17 s), Jev 95.7% at a 60% cutoff ($0.05, 0.15 s).
- Every model gained most from listing example data errors in the prompt; source context added at most a point for frontier models.
- Calibration: most of Jev's mistakes sit near 50% confidence; Luna's mistakes cluster at 90-100% confidence.
- Hybrid Jev (Opus reviews the 30-90% band, 22% of cases): 98.8% at $0.88. Hybrid Luna (10-90% band): 98.5% at $0.33.
- Harder test (twins, juniors and seniors, same names, major typos): Opus 95.1% ($5.39), hybrid Jev 95.3% ($2.94), hybrid Luna 91.6% ($1.15), Jev alone 89.1%, Luna alone 88.8%.
- Human review runs at about 50 identities an hour; estimating a rare over-merge rate within plus or minus 1% needs more than 1,000 samples.

## How they got it

Synthetic records from a simulated US town (University of Washington IHME's pseudopeople), assembled into labelled identities, graded by each model with the same prompt. Thresholds were tuned for accuracy, then cheap-model scores in a confidence band were forwarded to Opus. Costs and latencies are as observed in Hightouch's setup.

## Limits

- Synthetic data only, 1,000 identities per test; no real customer graphs.
- Thresholds were tuned on the same data they are reported on.
- Measures over-merging only, not under-merging; written by a vendor that sells identity resolution, with a hiring call to action.

## Why it matters for companies

A cheap model that knows when it is unsure, plus a frontier model for the uncertain cases, can match frontier accuracy at a fraction of the cost and spend more automatically when data get harder. The key selection criterion is calibration, not raw accuracy: Luna scored higher alone but was worse in the hybrid. Teams using LLMs to audit data quality, such as customer matching, should check where a model's errors fall on its confidence scale before designing the routing.

Original: https://hightouch.com/blog/llm-judges-identity-resolution