How Toyota Cut Ticket Processing From 60 min to 50 seconds
B · Useful, limited A Woven by Toyota security engineer describes a production agent with concrete design choices, token counts and costs, but results are self-reported from a couple of hundred tickets with no accuracy check against reviewer decisions.
Summary based on the full transcript·Original: about 25 minutes to watch
In short
A 25-minute Agentic AI Foundation talk (uploaded 4 November 2026) by Juan, a security engineer at Woven by Toyota, on an agent that pre-reviews exception tickets against about 300 internal security standards. The agent recommends approve, reject or "more information needed", and a person on the security assurance team still decides. It skips vector RAG: a CI job keeps a JSON of per-standard summaries, and the agent fetches full standards through two MCP tools. The title's "60 min" baseline is not stated in the talk.
Key numbers
- ~800k tokenssize of the roughly 300 internal security standards, about 2,000-3,000 pages
- about 99%usable answers after deterministic checks with up to three retries
- 45,000tokens per ticket, 95% of them spent reading standards
- about 25 secondsaverage processing time per ticket; most finish within 50 seconds
- 10 centsapproximate cost per run at GPT-4-class prices, excluding the summary build
As reported by the source.
English·Video·opens youtube.com
What it reports
- Corpus: about 300 standards, policies and procedures, 2,000-3,000 pages, about 800k tokens, already stored as Markdown in a GitHub repo.
- Preconditions named: standards in Markdown, approved LLM use with a budget, a ticketing API, and Slack as the only interface, so no new UI was built.
- Asking "does this comply?" always returns no, because an exception is non-compliant by definition; the output categories were designed around that.
- Retrieval: summaries plus two MCP tools (get summaries, get a standard). Fine-tuning and vector RAG were rejected as extra failure points. Built in Go to stay portable to a local model.
- Deterministic controls first: check that the tools were called, that retrieved text was pasted, and that the output format holds; retry up to three times. This gave about 99% usable answers; failures go to manual review with a Slack notice.
- Quality metrics: own checks for coherence and specificity, then Ragas relevance, faithfulness and context precision.
- Results: a couple of hundred tickets; about 45,000 tokens per ticket; most within 50 seconds, average about 25 seconds; about 10 cents per run.
- Main remaining problem per human feedback: precision (irrelevant standards retrieved), not recall.
How they got it
A practitioner talk with slides and Q&A, summarised here from the auto-caption transcript. Figures come from the team's own logs. The agent runs as a periodic CI batch over unprocessed tickets, not as a live service.
Limits
- No accuracy measured against final reviewer decisions and no reviewer time saved reported.
- The 60-minute baseline in the video title does not appear in the talk.
- Summary quality was never measured against the source standards; the system assumes the standards are consistent.
- Small volume (a couple of hundred tickets) from one team.
Why it matters for companies
Policy-heavy review work (security, compliance, procurement exceptions) is a good fit when the rules already live as clean text and a person keeps the decision. The useful pattern is cheap: summaries plus a fetch tool instead of a vector store, deterministic checks that the model actually read what it cites, and a retry loop before any human sees the output. Firms with policies trapped in PDFs or wikis should expect the preparation work to be the real cost.
Summary based on the full transcript. Read the original for the full argument and data.Watch the original on youtube.com