---
id: "lib-2026-10-22-deepseek-open-sources-its-inference-framework"
type: "library"
title: "DeepSeek open-sources its inference framework; gross margin estimated at 82%"
title_original: "DeepSeek开源推理框架，有人算出AI的毛利：82%"
lang: "ZH"
org: "机器之心 Jiqizhixin (DeepSeek)"
source_type: "Practitioner post"
date: "2026-10-22"
week: "2026-W43"
url: "https://nosov.ai/library/lib-2026-10-22-deepseek-open-sources-its-inference-framework/"
original_url: "https://www.jiqizhixin.com/articles/2026-10-22-2"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "Jiqizhixin news brief on DeepSeek's open-sourced inference engine that relays rare first-party serving figures; useful, but the headline 82% margin is an outside estimate and the article mostly restates DeepSeek's release notes."
summary_basis: "full-text"
original_minutes: "5"
tags: ["inference economics", "open source", "kv cache", "model serving"]
---

# DeepSeek open-sources its inference framework; gross margin estimated at 82%

Original title: DeepSeek开源推理框架，有人算出AI的毛利：82%

Tier B (Useful, limited): Jiqizhixin news brief on DeepSeek's open-sourced inference engine that relays rare first-party serving figures; useful, but the headline 82% margin is an outside estimate and the article mostly restates DeepSeek's release notes.

Summary based on the full text.

## In short

A Jiqizhixin article (22 October 2026) on DeepSeek's release, on 20 October, of DeepInfer, the inference engine behind its large-scale API service, plus deepseek-recipe, its request and token encoding layer, both under Apache 2.0. DeepSeek disclosed that one cluster serves 155 trillion tokens a day and earns $38 per card-hour at list prices. Commentators then estimated an 82% gross margin, which the article uses as its headline.

## Key numbers

- 155 trillion: tokens served per day by a single DeepSeek cluster, per DeepSeek
- $38: revenue per accelerator card per hour at list API prices, per DeepSeek
- 82%: gross margin estimated by outside commentators, assuming $7 an hour B300 rental and high utilisation
- 3 days / 7 days: KV cache retention for agent traffic and chat traffic, versus 5 minutes to 24 hours at other major providers
- 4-7%: throughput gain from dynamic-length speculative decoding versus fixed 5-token verification

## What it reports

- DeepInfer supports DeepSeek-V4.1-Flash, Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. It was originally built on vLLM, with ideas from SGLang and TensorRT-LLM, and covers optimised kernels, distributed parallelism, speculative decoding, async scheduling and constrained output.
- deepseek-recipe converts Chat Completions, Responses and Messages API formats into prompts and tokens and back, including reasoning content, tool calls and streaming, so third-party deployments behave like the official API.
- Full production still needs a KV cache system, vision encoding, separate prefill and decode deployment, and cross-replica load balancing, which are not all in the open release.
- KV cache kept 3 days for agent traffic and 7 days for chat on a separate 3FS storage cluster readable by all prefill machines. Anthropic, OpenAI and Google mostly keep caches 5 minutes to 24 hours.
- DSpark speculative decoding schedules verification by expected acceptance; dynamic length gives about 4-7% more throughput than verifying a fixed 5 tokens.
- Margin estimate: at $7 an hour per rented B300 and high utilisation, outside commentators put gross margin at 82%. Others on Hacker News argue for including full model lifecycle costs.
- The Information reported DeepSeek annualised revenue of about $1 billion, which the article equates to about 3,000 cards at $38 an hour.

## How they got it

The article summarises DeepSeek's repository and deployment notes, a Zhihu post, Hacker News discussion and a report in The Information, all linked. The 82% figure is a back-of-envelope calculation by third parties, not a DeepSeek disclosure.

## Limits

- The margin depends on assumed utilisation and rental price and excludes training and staff costs.
- Revenue per card-hour is at list prices; cache-hit input sells at one fiftieth of the uncached price, so traffic mix matters.
- A news brief: it adds context but no independent measurement.

## Why it matters for companies

The disclosed figures support the view that efficient inference can be profitable at low list prices, which bears on how long cheap API pricing will last. The multi-day KV cache for agent traffic is the most practical detail: long agent sessions that resend the same context cost far less where caches persist, so cache policy belongs in vendor comparisons. Firms considering self-hosting open models now have a production-grade engine to start from, with integration work still required.

## Key passages in translation

> 单个集群每天服务的吞吐量达到 155 万亿 token， 按 API 谷价算，整体收入达到每卡每小时 38 美元 。

A single cluster serves 155 trillion tokens a day; at API off-peak prices, total revenue reaches 38 dollars per card per hour.

> 官方披露说 Agent 流量保留 3 天，聊天流量保留 7 天 。

The official disclosure says agent traffic is kept for 3 days and chat traffic for 7 days.

> 有人算出了毛利率是 82%

Someone calculated the gross margin at 82%.

Original: https://www.jiqizhixin.com/articles/2026-10-22-2