---
id: "lib-2026-11-02-care-grounded-recursive-self-improvement-for"
type: "library"
title: "CARE: Grounded Recursive Self-Improvement for Production Customer-Service Agents"
lang: "EN"
org: "Meituan (Keeta)"
source_type: "Report"
date: "2026-11-02"
week: "2026-W45"
url: "https://nosov.ai/library/lib-2026-11-02-care-grounded-recursive-self-improvement-for/"
original_url: "https://arxiv.org/abs/2611.02839"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "Keeta (Meituan) and University of Melbourne authors give a well-documented production case with a concurrent incumbent control online, a frozen offline test, period-by-period tables and explicit limits; self-reported by the operator."
summary_basis: "full-text"
original_minutes: "90"
tags: ["ai technology", "business transformation", "customer service"]
---

# CARE: Grounded Recursive Self-Improvement for Production Customer-Service Agents

Tier A (Strong evidence): Keeta (Meituan) and University of Melbourne authors give a well-documented production case with a concurrent incumbent control online, a frozen offline test, period-by-period tables and explicit limits; self-reported by the operator.

Summary based on the full text.

## In short

A technical report (October 2026, arXiv 2 November) from the AI and Data team at Keeta, Meituan's overseas food-delivery service. The goal was to replace GPT-5.4 in after-sales customer service with an in-house model that keeps learning after deployment. A 24/7 inspection process reviews live conversations, separates model errors from faults in procedures, tools or harness, and trains only on the model's own errors. After five iterations the in-house model has fewer bad cases than the incumbent offline and parity online, at much lower cost.

## Key numbers

- 6.42% to 1.97%: offline inspection Bad Case rate of the in-house 31B model over five iterations; incumbent GPT-5.4 at 4.60%
- +1.73 points: online satisfaction of the final version vs the incumbent in the same period (p = 0.47, parity)
- 27.7%: reduction in median reply latency
- 70.5%: reduction in serving cost per service order after fully replacing the incumbent

## What it reports

- Deployment: seven regions (Saudi Arabia, Brazil, Hong Kong and four Gulf states), several languages; tasks include missing items, delays, refunds and handoff to staff.
- CARE-31B is post-trained from open-weight Gemma-4-31B-IT. After cold start on production traces it completes 93.0% of simulated cases but has a 6.42% Bad Case rate, worse than the untuned base (5.59%) and the incumbent (4.60%), and trails online satisfaction by 4.6 points.
- Five iterations bring the Bad Case rate to 1.97% on a frozen 559-case test; simulation accuracy ends at 94.10%, 2.7 points below the incumbent's 96.80%.
- Online, the gap closes: V4 +0.34 points, V5 +1.73 points against the concurrent incumbent; neither is significant, so the authors call it parity.
- Median reply latency falls 27.7%; serving cost per service order falls 70.5% after full replacement.
- Promotion needs a combined simulation-and-inspection score at parity, an online comparison and human approval.
- Training mixes failures from different procedure versions, each judged against the policy in force at the time.

## How they got it

Production traffic split between the in-house model and the incumbent in each period, with satisfaction from 4-5 star shares and Wilson intervals. Offline metrics from an executable simulation compiled from the current procedures and evidence-blinded inspection of its traces. Updates use on-policy self-distillation with the diagnosis and repair given only to the teacher.

## Limits

- Self-reported by the operating team; the simulator's fidelity to production is not measured.
- The Bad Case rate depends on screening recall and has no confidence intervals; early satisfaction periods have few ratings.
- Cost excludes research, storage, network and extra operations staff.
- Harness changes overlap with model updates, so the trend across versions is not the model's alone.

## Why it matters for companies

A mid-sized open model can replace a frontier API in a narrow, high-volume service, but only with an operating loop: continuous inspection, error attribution, a frozen test, an online control and a human sign-off. Cold start on historical traces was not enough and made the model worse than the base on bad cases. The cost saving is real but excludes the team that runs the loop, which is the actual investment.

Original: https://arxiv.org/abs/2611.02839