---
id: "lib-2026-10-08-can-ai-automate-epoch"
type: "library"
title: "Can AI automate Epoch?"
lang: "EN"
org: "Epoch AI"
source_type: "Report"
date: "2026-10-08"
week: "2026-W41"
url: "https://nosov.ai/library/lib-2026-10-08-can-ai-automate-epoch/"
original_url: "https://epoch.ai/publications/can-ai-automate-epoch"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "Epoch AI's head of benchmarks co-authors a new evaluation built on real in-house work tasks graded against employee standards, but it covers 11 tasks with one run per model and one grader."
summary_basis: "full-text"
original_minutes: "18"
tags: ["ai technology", "future of work", "task vs job automation"]
---

# Can AI automate Epoch?

Tier A (Strong evidence): Epoch AI's head of benchmarks co-authors a new evaluation built on real in-house work tasks graded against employee standards, but it covers 11 tasks with one run per model and one grader.

Summary based on the full text.

## In short

Epoch AI published the first Epoch Automation Reports on 8 October 2026, written by Kelly Hong and Greg Burnham. Six models were given real Epoch work (graphics, Data Insights, data explorers, data center research, research design) with the context a new hire would get. Claude Fable 5.1 and GPT-6 Astra lead and handle well-defined steps reliably, but neither can produce Epoch-quality work end to end. The authors conclude AI cannot yet replace workers, at least not at Epoch.

## Key numbers

- 11 tasks: real Epoch work tasks across five categories in the launch suite
- 6 models: frontier and open-weight models tested, each in its own agent harness at maximum reasoning
- 61 of 280: responses cut off by a 4096-token budget in GPT-6 Astra's own pilot, later presented as a finding
- 158: Kimi K3 score on the Epoch Capabilities Index, roughly tied with Grok 4.6 despite failing basic tasks

## What it reports

- Task suite: 11 tasks in five categories; models tested on Codex, Claude Code, Grok Build, Antigravity, Kimi Code and Qwen Code at their highest reasoning settings.
- Fable 5.1 and GPT-6 Astra are broadly tied on aggregate score; their lead comes from coding, computational analysis and computer use.
- One graphic design task (restyling a Matplotlib chart to house style) was removed because Fable 5.1 produced Epoch-quality output; Epoch now largely automates that step.
- No frontier closed-weight model made a factual error in its Data Insights. Kimi K3 built a whole insight on a filter error, counting all arXiv papers instead of physics papers.
- Kimi K3 scores 158 on the Epoch Capabilities Index, about the same as Grok 4.6, yet fails basic tasks Grok 4.6 handles; the authors read this as a gap that benchmarks miss.
- Models missed implicit standards despite reference material: Fable 5.1 produced far denser graphics than house style; GPT-6 Astra chose a topic too niche for Epoch's audience.
- In a research design task, GPT-6 Astra used a 4096-token budget that cut off 61 of 280 responses, then reported the effect as "sensitivity to the acquisition budget".
- Three of six models (Fable 5.1, Gemini 3.8 Flash, Grok 4.6) picked the same topic from Epoch's polling data.

## How they got it

Tasks were drawn from Epoch's own work, with tools such as Figma, Google Drive and satellite imagery ordering sites, and full permissions in a dedicated workspace. Rubrics combining objective and subjective criteria were built with Epoch staff expert in each task. Each output was graded by a single human; the authors weight qualitative observations from outputs and agent trajectories above the scores.

## Limits

- One run per model per task and a small suite; the authors say scores are noisy.
- Subjective grading by one grader against Epoch-specific standards, which may not transfer to other work.
- Each model ran in its own harness, so some differences may reflect tooling rather than the model.

## Why it matters for companies

The pattern matches what managers see in pilots: frontier models execute defined steps well and fail on house standards, audience judgment and experimental design. Giving a model the same reference material as a new hire does not transfer tacit conventions. Benchmark rankings can hide large differences on real tasks, so internal evaluations on actual work are worth building. The removed design task shows where automation already pays: narrow, well-specified steps with clear reference components.

Original: https://epoch.ai/publications/can-ai-automate-epoch