Built to Remember, Shipped to Forget: Operational Contracts, Failures, and Repairs in Containerized LLM Agent Deployments
A · Strong evidence A software-engineering study from UNC Wilmington and Oakland University that checks 433 real agent deployments against contracts derived from first-party documentation, confirms predictions by fault injection and mines developer repairs, with inter-rater agreement and a replication package.
Summary based on the full text·Original: about 45 minutes to read
In short
A paper (arXiv, 25 October 2026) by Meriem Mastouri, Emna Ksontini (University of North Carolina Wilmington) and Amine Barrak (Oakland University). Container platforms treat a container as disposable: stop it, replace it, discard what is inside. Agent frameworks keep conversations, pause runs for human approval and resume interrupted work from saved state. Neither side states what a deployment owes the other. The authors write those obligations down as 14 "operational contracts" and find that many real agent deployments break them, so agents forget conversations or lose paused work when the platform does routine maintenance.
Key numbers
- 433deployed open-source LangGraph agent systems assessed
- 37.4%systems that need durable state but lose it on a container restart
- 41.5%the same systems losing state when the container is replaced
- 14 of 16predicted contract violations that became runtime failures under induced events
- 78%developer repairs that change application code rather than deployment files
As reported by the source.
English·Web page·opens arxiv.org
What it reports
- At least one contract applies to 75.5% of the 433 systems; violation rates per contract range from 11% to 100%.
- Of systems that must keep state beyond the process, 37.4% lose it on restart and 41.5% on container replacement.
- Most systems that lose state on restart (77 of 99) ship no persistent store at all; 16 of 76 systems using an in-memory checkpointer deploy a persistent store next to it without routing agent state there.
- Every assessable deployment stops LangGraph's production server in the middle of draining running work.
- Induced events on 12 systems turned 14 of 16 predicted violations into failures; all seven predicted holds passed.
- From 39,261 commits, pull requests and issues the authors found 122 repairs with 35 recurring fix patterns. 78% change application code, and only about two in three fully restore the guarantee.
- The framework warns that resuming a run may repeat completed steps, which can mean sending an email or charging a payment twice.
How they got it
Contracts were derived from LangGraph, Docker, Compose and Kubernetes documentation, with LLM-assisted extraction and author validation. The corpus came from GitHub repositories with at least 10 stars that use LangGraph and ship a Dockerfile, pinned to a 6 July 2026 snapshot and annotated with a codebook (agreement kappa about 0.71 to 0.94). Seven platform events were induced on 12 selected systems through a fault-injection harness. Repairs were mined from repository histories.
Limits
- One framework (LangGraph) and three platforms; other stacks may produce different contracts.
- Public Python repositories only, many of them demos or small projects; enterprise deployments may differ.
- The 12 runtime systems were chosen for coverage, so they confirm that failures occur, not how often.
- Behaviour hidden inside deployed images can escape repository-based checks.
Why it matters for companies
Agents that wait for a human approval or run over days depend on state surviving restarts, upgrades and node moves, which platforms perform as a matter of course. This paper shows the gap is common and usually fixed in application code after something breaks. Any agent that pauses for sign-off or touches payments needs a deployment review against a checklist like these 14 contracts: durable state store, graceful stop, safe resumption without repeating side effects.
Summary based on the full text. Read the original for the full argument and data.Read the original on arxiv.org