---
id: "lib-2026-09-28-periodic-weak-spots-phase-sensitivity-from"
type: "library"
title: "Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression"
lang: "ZH"
org: "ByteDance Seed (arXiv)"
source_type: "Paper"
date: "2026-09-28"
week: "2026-W41"
url: "https://nosov.ai/library/lib-2026-09-28-periodic-weak-spots-phase-sensitivity-from/"
original_url: "https://arxiv.org/pdf/2609.36322"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "A frontier-lab paper from ByteDance Seed with controlled pretraining experiments, causal interventions and tests on released DeepSeek models, reporting a measurable failure mode that average benchmarks hide."
summary_basis: "full-text"
original_minutes: "120"
tags: ["ai technology", "benchmarks", "long context evaluation"]
frames: ["verification-capacity", "tacit-to-explicit"]
---

# Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression

Tier A (Strong evidence): A frontier-lab paper from ByteDance Seed with controlled pretraining experiments, causal interventions and tests on released DeepSeek models, reporting a measurable failure mode that average benchmarks hide.

Summary based on the full text.

## In short

A September 2026 arXiv paper from ByteDance Seed (Xingyu Zhu, Pu Yi, Ziheng Cheng, Ang Lv and colleagues, with interns from Princeton, Stanford and Berkeley). Models that compress the KV cache in fixed-size chunks, as DeepSeek-V4 does, retrieve the same fact well or badly depending on where it falls relative to chunk boundaries. The authors call this phase sensitivity and show average benchmark scores conceal it. The paper is published in English.

## Key numbers

- 40.2 pp: largest gap in 128K-context retrieval accuracy across token phases in DeepSeek-V4 base models
- 9.9% vs 58.4%: worst-phase retrieval accuracy, compressed W8/S8 model vs full-attention baseline with similar mean accuracy
- up to 78 pp: phase gaps in compressed pretrained models, against at most 6.1 pp for full-attention baselines

## What it reports

- In DeepSeek-V4-Flash-Base, a code completion flips between the correct "8" and the wrong "32" with a period of four tokens as filler text is lengthened, matching the compression stride of 4.
- Needle-in-a-haystack test: 128K-token prompts with 16k key-value pairs, sorted into eight groups by position modulo eight.
- Base checkpoints show gaps up to 40.2 percentage points between best and worst phase; DeepSeek-V4-Pro-Base 34.77 pp; post-trained V4-Flash 19.14 pp and V4-Pro 14.84 pp.
- DeepSeek-V4.1-Flash, evaluated on 2,560 prompts per group, has 92.38% mean accuracy and a 6.09 pp gap.
- Every compressed model the authors pretrained (Qwen3-0.6B backbone) shows periodicity matching the stride; full-attention baselines stay flat.
- Accuracy varies by more than 50 pp even among phases where key and value share one window, so window boundaries alone do not explain it.
- Knockout experiments show different KV heads specialise in different phases ("phase specialization"); a theoretical model shows gradient flow can produce this.

## How they got it

Tests on five DeepSeek-V4 family checkpoints, then pretraining small transformers from scratch with KV compression as the only major architectural change, varying window and stride. Causal interventions (mean replacement on heads and layers) locate which components serve which phase. An idealised retrieval model supplies the theory.

## Limits

- Mechanistic conclusions are strongest in small controlled models and are not shown to be the universal cause in large models.
- Effects on real tasks such as refusals or long-document work are flagged as future work.
- Authors work at a lab competing with DeepSeek.

## Why it matters for companies

Companies selecting long-context models for document review or retrieval usually compare average scores. This paper shows a model can score well on average and still miss specific facts depending on token position. Procurement and testing teams can add a simple check: shift the same input by a few tokens and see whether answers change. The issue matters most for cheaper long-context models that use compression to cut cost.

## Key passages in translation

The paper is written in English by a Chinese lab, so the passages below are quoted in the original language; no translation is needed.

> "the same information can be easy to retrieve at one phase and difficult at another."

The core claim: retrieval quality depends on where information sits relative to compression windows.

> "high average accuracy can coexist with systematic positional failures."

Why standard benchmark averages are insufficient for these models.

> "Their reliability should also be measured across phases, for example by shifting the input by a few tokens while keeping the queried information fixed."

The practical test the authors recommend.

Original: https://arxiv.org/pdf/2609.36322