Building Legal Agents When You Can't See the Data - Jacob Lauritzen, CTO of Legora
B · Useful, limited Legora's CTO describes how a legal-agent vendor evaluates and builds without reading client data; concrete practice, but every figure is his estimate on a friendly podcast with no customer outcomes.
Summary based on the full transcript·Original: about 67 minutes to listen
In short
A 67-minute Latent Space episode (2 November 2026) with Jacob Lauritzen, CTO of Legora, a legal AI vendor founded in Stockholm. Legora cannot read client traces, so it evaluates agents in simulated worlds and synthetic data rooms, and learns from failures only through privacy-safe categories. He describes the move from assistant to agent, a table-based review product, and why law-firm economics slow codification of knowledge.
Key numbers
- 70% to 99%correct citations on one problem after a single model release, with no change on Legora's side
- three engineers and three lawyerssize of the evaluation team, plus a pool of legal engineers
- 10 or 20failure types that paraphrased traces are classified into
- 20 to 100engineers: the team size once said to be enough forever vs today
As reported by the source.
English·Video·opens youtube.com
What it reports
- A citation problem went from about 70% to about 99% correct with one model release.
- Evaluation team: about three engineers and three lawyers, plus "legal engineers" with three to four years in big law, often with a CS degree. Tests run in simulated environments, including a synthetic data room of 10,000 files.
- Traces are paraphrased into safe categories and failures classified into 10 or 20 known types rather than read raw.
- Tabular review asks the same question of every document (for example 10,000 employment contracts), with citations to source and cell-level review flags; customers work across up to a million documents.
- The agent drafts and edits Word documents, runs diligence and coordinates work between lawyers, clients and other agents; Legora used it on its own acquisitions.
- Partners lack incentives to codify knowledge; the billable hour shifts time rather than lowering bills.
- Remaining barriers to more autonomy: cost, latency, and "two out of 100 times" the agent does something unwanted.
- Engineering headcount grew from a planned 20 to about 100.
How they got it
Interview. All figures are the speaker's estimates; transcript from auto-captions.
Limits
- Vendor CTO on a friendly show; no customer outcome data or accuracy figures beyond the citation anecdote.
- The "two out of 100" rate is a remark, not a measurement.
Why it matters for companies
When the data cannot be seen, evaluation has to be built from synthetic environments and coarse failure categories, which is the situation of most internal teams handling confidential documents. A mixed team of domain experts and engineers owning evals is a model worth copying. Improvements also arrive from model releases alone, so rerun evals when models change before rebuilding anything.
Summary based on the full transcript. Read the original for the full argument and data.Watch the original on youtube.com
- Tags
- Future of work