---
id: "t-2026-10-10-the-tasks-worked"
type: "thought"
title: "The tasks worked. The job did not."
dek: "Epoch AI tested six models on its own work. Where they failed is where most companies run on unwritten rules."
date: "2026-10-10"
week: "2026-W41"
url: "https://nosov.ai/writing/the-tasks-worked-the-job-did-not/"
linkedin: "https://www.linkedin.com/feed/update/urn%3Ali%3Ashare%3A7514626304940421120"
frames: ["tacit-to-explicit", "specification", "verification-capacity"]
author: "Ivan Nosov"
---

# The tasks worked. The job did not.

Epoch AI tested six models on its own work. Where they failed is where most companies run on unwritten rules.

The question I keep coming back to: how do you get AI to do the whole job, beyond the individual tasks, and do it consistently at scale?

Epoch AI published a useful reference on 8 October. The research group, which measures how fast AI capabilities grow, tested six models, including Fable 5.1 and GPT-6 Astra, on real tasks from its own work - charts, analysis, research design. The models got the context a colleague would get and no human help, and the results were graded against Epoch’s own staff standard.

The tasks worked. Coding, analysis and operating a computer were reliable, and better than Epoch expected.

The job did not. The models failed in three places:

1. Epoch's own standards: even with many examples, they missed the conventions a new colleague learns quickly, like the visual style and what the audience cares about.

2.. Designing experiments: good research ideas, but tests that did not measure what they claimed.

3. Their own mistakes: flawed setups presented as key findings.

For me this is a useful map of where the work is. The first gap is the one I think about most, because every company runs on it: how the month-end report should look, which supplier always gets an exception, what a hiring manager really looks for. People learn this by watching colleagues, while a model only works with what it is given.

So the way to the whole job goes through making those rules explicit, giving them an owner and keeping them current, plus building checks for the other two gaps, because a model will not catch its own flawed setup.

That is the organisational capability everyone talks about, broken down into work that can actually be done.

## Sources

1. [EN] Can AI automate Epoch?. Epoch AI, 2026-10-08. https://epoch.ai/publications/can-ai-automate-epoch