Karpathy’s march of nines

One concept from Andrej Karpathy's latest interview that I take as a large takeaway is The "March of Nines".

I just listened to Andrej Karpathy's 2+ hour conversation with Dwarkesh Patel, and it's exceptional - frank, revealing, and full of insights into how frontier AI actually gets built.

Link to the video: https://lnkd.in/d-Q-spcK

For context: Karpathy founded OpenAI's computer vision research, led AI at Tesla for 5 years building their self-driving system from scratch, and now runs Eureka Labs. This is someone who's actually built the technology, not someone speculating about it.

The interview covers everything from why LLMs are "collapsed," to education post-AGI, to why we're "summoning ghosts, not building animals." But there's one concept I wanted to share because it's rarely discussed yet deeply relevant to how businesses should think about AI deployment:

The "March of Nines" 🎯

Getting a system to work 90% of the time (demo quality) is just the FIRST nine.

Getting it to 99%? That's the second nine - and requires roughly the SAME amount of work.

Then 99.9%, 99.99%, and so on. Each nine demands comparable effort.

During his 5 years at Tesla, Karpathy estimates they progressed through only 2-3 nines. Self-driving still isn't fully solved.

Why this matters for business strategy:

Most AI discussions in business focus on capability: "Can the AI do this task?" But Karpathy's framework reveals the real question: "Which nine do we need, and are we prepared for that journey?" 🤔

Software engineering is his example. The cost of failure is high - bugs expose data, vulnerabilities topple systems. A demo coding agent that works 90% might write a simple app. But production-grade code for millions of users? That requires several more nines and potentially years of iteration.

He saw this at Waymo: a "perfect" demo in 2014, then a decade to limited deployment. Demos hide in narrow conditions. Production faces "all the corners of reality" - edge cases, unusual contexts, failure modes you never anticipated. Even today, Waymo has humans in remote operation centers providing support.

How to think about AI adoption 💡

The march of nines helps you be strategic:

Identify which reliability threshold your use case actually requires

Budget time and resources for the journey between nines, not just the demo

Start with lower-stakes applications where 90-95% might be sufficient

Design human-in-the-loop systems for critical processes

Stay curious about what breaks and why - that's how you reach the next nine

Karpathy's advice throughout: don't write blog posts, don't make slides. Build it. Get it to work. That's how you discover what you're missing. That's how you push to the next nine. 🔨

It's packed with insights like this - the kind that only come from someone who's actually walked the path. 🎧