2 sources filed · RU 1 · JA 1
Data: five fictional projects and eight fictional engineers (40 pairs), drafted by Claude and reviewed by a person, including hard cases such as an overqualified tech lead and a vague self-assessor.
0.87 / 0.78 / 0.47Kendall tau-b against Opus 5.5 labels for Jev, Haiku 4.5 and the ruri embedding model
Modelled case: 40,000 contacts a month, 90% routed to the agent (36,000 dialogues), three LLM calls per dialogue, about 12,000 input and 800 output tokens, a guard model, RAG and order-status tool on Cloud.ru services.
25x to 1.4xtoken cost gap between the cheapest and dearest model per dialogue, and what remains of it per useful answer (51 to 73 roubles)
1 source filed · JA 1
Workload: synchronous and batch jobs reading text and images, about 40 million tokens a day.
about 40 milliontokens a day handled by Zenn's translation, review and spam-detection AI features