---
id: "lib-2026-10-13-why-a-former-store-clerk-ranked-near-the-top-of"
type: "library"
title: "Why a former store clerk ranked near the top of Krafton's AI ability test"
title_original: "크래프톤 AI 역량 평가에서 개발 경험 없는 매장 직원 출신이 상위권에 오른 이유"
lang: "KO"
org: "Builder Josh (HOW I AI)"
source_type: "Podcast"
date: "2026-10-13"
week: "2026-W42"
url: "https://nosov.ai/library/lib-2026-10-13-why-a-former-store-clerk-ranked-near-the-top-of/"
original_url: "https://www.youtube.com/watch?v=CLpN3zl8noQ"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "Builder Josh podcast with the two Krafton AI Frontier builders of the Probe hiring assessment; a detailed first-hand design account, but validity evidence is one internal hire and hackathon anecdotes."
summary_basis: "transcript"
original_minutes: "48"
tags: ["hr technology", "ai skills assessment", "recruiting"]
---

# Why a former store clerk ranked near the top of Krafton's AI ability test

Original title: 크래프톤 AI 역량 평가에서 개발 경험 없는 매장 직원 출신이 상위권에 오른 이유

Tier B (Useful, limited): Builder Josh podcast with the two Krafton AI Frontier builders of the Probe hiring assessment; a detailed first-hand design account, but validity evidence is one internal hire and hackathon anecdotes.

Summary based on the full transcript.

## In short

An episode of the Korean HOW I AI podcast on the Builder Josh channel (13 October 2026, 48 minutes) with Koo Han-mo (product owner, ex-Bain) and Cho Jeong-seok (AI product engineer, ex-Naver Z) from Krafton's AI Frontier division. They built Probe, the first product of Krafton's "AI-native talent" platform Cofa. It replaces coding tests with a work simulation in which candidates use Claude Code or Codex, question stakeholder agents about a deliberately vague brief, and get scored by a panel of agents on how they work with AI.

## Key numbers

- 20+: evaluator agents running in parallel to score one candidate's task
- about 6 hours: length of the hackathon version of the Probe work simulation
- ~95 vs low 70s: calibration scores when models take the test: Opus about 95, Haiku in the low 70s, Sonnet between
- about 2 years: store experience of the non-developer candidate who reached the top tier at a hackathon

## What it reports

- Distinction between AI-literate people, who speed up their existing workflow, and AI-native people, who redesign the workflow from the goal, including building a tool instead of buying one.
- Environment: IDE-like workspace with a terminal running Claude Code or Codex, browser preview, option to upload one's own skills; the brief is modelled on a real Krafton task.
- Stakeholder agents answer questions but will not hand over the solution; candidates who build context through small questions score well, while "tell me the four things I need to do" is deflected.
- More than 20 agents score each candidate on three axes, taken from a paper on technical, intent and cognitive debt: output quality and architecture; how the candidate steers AI; and whether they understand and can defend their own output.
- Token and cost use is tracked; the better candidates spend less.
- Calibration: when models take the test, scores follow benchmark order (Opus about 95, Haiku low 70s).
- At a hackathon with CJ Olive Young, a non-developer with about two years in an Olive Young store reached the top tier. Probe has been used for one internal hire and is planned for Krafton's open recruitment and an SK hynix hackathon.

## How they got it

Interview with the product's builders. Transcript from Korean auto-captions. Outcomes are the speakers' own accounts.

## Limits

- No predictive validity data: no link yet between Probe scores and later job performance.
- No information on fairness checks, candidate experience or adverse impact.
- The builders are promoting their own product.

## Why it matters for companies

For HR and hiring teams this is a concrete design to borrow from: assess people doing real, ambiguous work with AI tools, score how they question, decompose and verify, and test whether they understand what the AI produced. Cost per candidate as a signal is a simple, cheap addition. The speakers' advice to HR is the right first step: agree internally what "AI ability" means, ideally by taking the test yourselves. Before using such tests in selection decisions, firms need their own validity and fairness evidence, which this source does not provide.

## Key passages in translation

> 저희 개발자 리드분께서 지금은 코드를 사랑하는 사람은 뽑고 싶지 않다라고 하시더라고요.

Our lead developer said that right now he doesn't want to hire people who love code.

> 사실 되게 재밌는 거는 잘하시는 분들일수록 비용 적게 쓰시거든요.

What's actually really interesting is that the better people are, the less they spend.

> 그 한 명의 과제를 평가하기 위해서 20개가 넘는 에이전트가 병렬적으로 돌아가거든요.

To evaluate one person's task, more than 20 agents run in parallel.

Original: https://www.youtube.com/watch?v=CLpN3zl8noQ