---
id: "lib-2026-10-13-aiops-holmesgpt-on-three-projects"
type: "library"
title: "AIOps: HolmesGPT on three projects"
title_original: "AIOps: наш опыт внедрения HolmesGPT на трёх проектах"
lang: "RU"
org: "Tuna (Habr)"
source_type: "Practitioner post"
date: "2026-10-13"
week: "2026-W42"
url: "https://nosov.ai/library/lib-2026-10-13-aiops-holmesgpt-on-three-projects/"
original_url: "https://habr.com/ru/articles/1091654/"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "DevOps team Tuna's detailed month-one report on running the open-source HolmesGPT agent across six Kubernetes clusters, with an open-sourced bridge, a 52-run model comparison and costs reconciled to billing; the headline accuracy figure is openly labelled an impression."
summary_basis: "full-text"
original_minutes: "20"
tags: ["ai technology", "aiops", "sre agents", "data leakage risk"]
---

# AIOps: HolmesGPT on three projects

Original title: AIOps: наш опыт внедрения HolmesGPT на трёх проектах

Tier A (Strong evidence): DevOps team Tuna's detailed month-one report on running the open-source HolmesGPT agent across six Kubernetes clusters, with an open-sourced bridge, a 52-run model comparison and costs reconciled to billing; the headline accuracy figure is openly labelled an impression.

Summary based on the full text.

## In short

A Habr post (13 October 2026) from Tuna, a DevOps services team, on putting HolmesGPT, a CNCF Sandbox SRE agent, behind Alertmanager in six Kubernetes clusters: its own and five client clusters across a Russian cloud and a foreign two-cloud project. The agent has read-only access and only writes a hypothesis into Slack or Telegram a few minutes after an alert; it fixes nothing. Because the open-source version has no push integration, the team built and published holmes-alert-bridge, a Go service whose main job is to cap spend.

## Key numbers

- 645 to 41: alerts delivered to the bridge in Tuna's first month versus those that reached the agent after dedup, allowlist and throttling
- 52: test runs across eight models in three waves, including a task with a known answer
- 10-30 cents: cost of one alert investigation, depending on model; 2-3 minutes median duration
- 90-95%: share of hypotheses the team believes are correct, an on-call impression rather than a measurement
- ~90%: estimated share of input tokens served from cache, which carries the cost economics

## What it reports

- Median time from alert firing to a hypothesis in chat is about four minutes including Alertmanager grouping delay.
- Bridge controls: allowlist by alert name (empty list blocks start-up), dedup per episode, default cap of two investigations an hour and ten a day, one worker, 15-minute timeout that cuts the stream.
- Model bake-off: deepseek-v4-flash completed 16 of 16 runs and solved the known-answer task 4 of 4; half the candidates failed it the same way, confidently stating that a managed autoscaler did not exist.
- Variance within one model exceeded the price difference between models; single-run comparisons were misleading.
- Production results over 30 days to 6 October 2026: 22 of 22 successful investigations in Tuna, 92.9% in the Russian-cloud project, 15 of 15 abroad on Claude Sonnet 5. Monthly bills were a few dollars per project; Tuna's estimate matched provider billing to the cent.
- A 200k-token window triggered compaction near 126k and made runs slower and costlier; a 400k window stopped it.
- A "have we seen this before" skill searched the wrong log field in two of three setups for a month and always reported no history, including for an episode investigated 19 times.
- Tools not named in a skill were almost never called.

## How they got it

Production deployment notes with configuration, logs and billing. The model comparison used 4 to 16 runs per model, priced as an upper bound without cache. The team states it has no clean before-and-after comparison for the model switch, because prompts and the bridge changed at the same time.

## Limits

- Hypothesis accuracy has not been measured against real incident causes; the 90-95% is a stated impression.
- No measurement yet of time saved for on-call staff.
- Small samples per model in the bake-off; the Sonnet project had no model comparison.

## Why it matters for companies

This is a useful template for putting an agent into operations without giving it write access: read-only RBAC, hypothesis-only output, hard spend caps outside the model, and a monthly limit on the provider key. The data warning applies to any agent that reads logs: whatever it reads goes to the model provider, so tokens and personal data in logs must be fixed in the application first, and zero-retention settings must be verified with live requests. The broken memory skill is the strongest lesson: an agent tool that returns nothing looks the same as one that works, so skills need testing like code.

## Key passages in translation

> По нашему опыту гипотеза оказывается верной в 90–95 % случаев. Это именно оценка по ощущениям от дежурств, а не результат замера

In our experience the hypothesis turns out correct in 90-95% of cases. This is an estimate from how on-call shifts felt, not the result of a measurement.

> Агент не отличает секрет от обычной строки: содержимое лога попадает в запрос к модели как есть.

The agent does not tell a secret from an ordinary string: the log content goes into the request to the model as is.

> Мораль шире: skill для агента надо тестировать как код.

The moral is wider: a skill for an agent has to be tested like code.

Original: https://habr.com/ru/articles/1091654/