Woowa Brothers' AI governance, built into the platform (우아한형제들의 AI 거버넌스, 플랫폼에 담다)
A · Strong evidence Woowa Brothers' AI platform team documents a production governance design end to end (gateway, request and approval flow, agent controls, platform isolation) in enough detail to copy, though the only measured outcome is one before-and-after caching result.
Summary based on the full text·Original: about 15 minutes to read
In short
A 6 November 2026 post on the Woowahan tech blog by Lee Jun-su of the AI platform team at Woowa Brothers (operator of the Korean delivery app Baemin). All LLM and agent calls go through one LiteLLM-based AI Gateway; an in-house AI Studio handles registration, policy evaluation and approval, issuing project keys automatically when a request is inside default limits and routing only exceptions to cost and security reviewers. Agents inherit the same controls plus three more: who may call them, which tools they may use, and whose permissions they run under. The platform itself sits in a separate AWS account and VPC.
Key numbers
- 5% to 97%prompt-cache rate of one project after moving per-request content out of the system prompt
- -84%cost per request for that project after caching
- 0.79 s to 1.33 saverage response time for the same project, raised by an extra gateway cache lookup
- about 10%lower token price on Bedrock Global versus Geographic cross-region inference
- 2 dayslag before Bedrock billing data appeared, the reason cost is computed at the gateway per call
As reported by the source.
Korean·Web page·opens techblog.woowahan.com
What it reports
- The project is the unit of governance: owner, budget, credentials and policy attach to it. Virtual keys carry no settings of their own and inherit the project's allowed models and budget; provider keys never reach requesters.
- Each gateway call passes four checks: key, allowed model or tool, budget and RPM/TPM limits, then forwarding with retry or fallback model. Cost and traces are logged per project.
- Three separate request channels (Bedrock, external providers, gateway) were merged into AI Studio. Policies are stored as files, so the same request always gets the same verdict and the policy version is recorded with the approval.
- Default scope (for example in-house models) gets an immediate key; expensive models or budgets above a threshold trigger parallel cost or budget review; forbidden data or models are blocked before issue.
- Requests with sensitive data are routed to self-hosted models in the company's own data centre; general data may go to Bedrock, Azure or GCP. PII guardrails can MASK (and restore in the response) or BLOCK, at pre-call, during-call or post-call stages.
- Measured case: one project's cache rate went from 5% to 97%, cost per request fell 84%, and average response time rose from 0.79 s to 1.33 s.
- Agents: new agents start with no allowed channels or users; only security-reviewed MCP servers can be connected; new MCP tools arrive switched off; Text-to-SQL queries run under the requester's account. A shared Helm chart fixes call paths and restricts terminal, file, code execution and browser tools.
- Platform isolation used five criteria (issues credentials, concentrates permissions, talks to external AI, enforces policy for many services, spans environments); direct provider calls bypassing the gateway are blocked company-wide.
- Next steps: agent registry, per-session cost limits, agent authentication, wider guardrails and auto-approval, and quality evaluation from Slack feedback.
How they got it
A first-hand architecture account by the team that built it, with diagrams and example screens. Apart from the caching before-and-after, results are described rather than measured. The author states that the dashboard figure showing a project at 45% of spend and 92% of budget is illustrative, with names, costs and ratios not real.
Limits
- Self-reported by the platform team; no figures on number of projects, total spend, review times or incidents prevented.
- The only measured outcome is one project's caching change.
- No quality evaluation yet: the author says usage and cost cannot show whether a project helps.
Why it matters for companies
This is a concrete template for running AI governance as platform engineering rather than as a queue of approvals. The useful moves: one mandatory call path before usage grows, policy as versioned files, automatic issue inside default limits, review only for exceptions, real-time cost at the gateway instead of waiting for invoices, and agents that act with the requester's permissions. The caching result also shows that cost work is a per-project habit, and that it can trade latency for price.
Key passages in translation
Korean original
사용을 막으면 비용과 보안 위험은 줄지만, AI로 얻을 성과도 같이 사라집니다.
Blocking use lowers cost and security risk, but the results AI could deliver disappear with it.
Korean original
기본 범위는 플랫폼이 확인하고 바로 발급하고, 사람은 예외 항목만 검토합니다.
The platform checks the default scope and issues keys immediately; people review only the exceptions.
Korean original
그래서 Agent에게 권한을 따로 주지 않고, 묻는 사람의 권한으로 실행합니다.
So the agent is given no permissions of its own; it runs with the permissions of the person asking.
Summary based on the full text. Read the original for the full argument and data.Read the original on techblog.woowahan.com