Turning log-definition criteria in people's heads into an AI skill for everyone
머릿속에 있던 로그 정의 기준을 모두의 AI 스킬로 만들기
B · Useful, limited Toss DataOps managers, non-developers, describe turning tacit log-definition know-how into a shared AI skill with a clean split between model judgement and machine-checked rules; a clear practice case, but with no adoption, time or accuracy figures.
Summary based on the full text·Original: about 7 minutes to read
In short
A Toss tech blog post (23 October 2026) by Park Ju-young, Kim Seo-won and Jo Ae-ra, DataOps Managers who define and maintain app-log specifications at Toss. Rules lived in an "app-log encyclopedia", but domain context lived in individual managers' heads. One manager built a personal AI skill; the team turned it into a shared skill now shipped inside the log platform's CLI, lctl. The skill proposes specs and reuse candidates; machine-checkable rules stay in the CLI; people confirm before anything is written.
Key numbers
- 4requirements every log definition must meet: format, domain continuity, analytical need only, policy and safety
- 3problems the evaluation surfaced: finding existing logs, naming mismatches across domains, managing domain knowledge
- 1new log a colleague had to create in one task, with nearly everything else reused from existing specs
As reported by the source.
Korean·Web page·opens toss.tech
What it reports
- Log definitions must follow the set format, keep domain continuity, record only what analysis needs, and respect policy and safety.
- The skill is not a spec generator. It checks the encyclopedia standard, searches the domain's existing specs fully for reuse, proposes a spec from the screen design, and writes to the system only after the manager confirms. It does not read user log data.
- Division of labour: the skill handles context-dependent judgement and asks people when unsure;
lctlchecks machine-decidable rules such as name format before saving; the Log Center rejects non-compliant specs. Format rules are queried fromlctlat run time rather than written into the skill, so rule changes apply automatically. - One long document holding all rules and procedures caused the model to miss important rules; splitting it into task-specific documents, with the skill pointing to each, fixed this.
- Process change: business users built the tool themselves, and the platform team reviewed it and shipped it in
lctl, instead of the old request-and-wait cycle. - Benefits reported: fewer duplicate parameters (for example, avoiding a new
issuerfield whencard_companyalready existed), shared standards across managers, and more time spent on whole user flows. - Evaluation: managers re-defined screens already specified in their domains and compared the output with the existing spec. Domain context, more than rules, determined quality. Fixes added category inference, similar-name checks and domain standard inputs; a domain knowledge base is next.
How they got it
Practitioner account with colleague quotes and an informal evaluation against existing specs used as answer keys. No quantitative results are given.
Limits
- No figures on time saved, accuracy against answer keys, adoption or error rates.
- Testimonials come from internal colleagues.
- One company's log platform; the CLI integration is specific to Toss.
Why it matters for companies
This is a small, well-structured example of moving tacit operational know-how into a shared skill. Three design choices transfer: keep deterministic rules in tools that enforce them, let the model handle judgement and ask when unsure, and split long instruction documents by task. It also shows a different operating model between business teams and platform teams: domain experts build the tool and the platform team hardens and distributes it.
Key passages in translation
Korean original
그렇게 만든 '로그 정의 스킬'은 로그 명세를 대신 찍어내는 생성기는 아니에요.
The "log definition skill" we built is not a generator that stamps out log specs on our behalf.
Korean original
처음엔 한 문서에 규칙과 절차를 다 담았는데, 길어질수록 AI가 중요한 규칙을 놓쳤거든요.
At first we put all the rules and procedures in one document, but the longer it got, the more the AI missed important rules.
Korean original
평가를 거듭할수록 분명해진 건, 정의의 품질을 좌우하는 게 규칙보다 도메인의 맥락이라는 점이었어요.
The more we evaluated, the clearer it became that what decides the quality of a definition is domain context more than rules.
Summary based on the full text. Read the original for the full argument and data.Read the original on toss.tech