---
id: "lib-2026-10-15-small-language-model-serving-for-search-and"
type: "library"
title: "Small language model serving for search and recommendation"
title_original: "搜推场景小语言模型在线推理工程实践"
lang: "ZH"
org: "Meituan 美团"
source_type: "Practitioner post"
date: "2026-10-15"
week: "2026-W42"
url: "https://nosov.ai/library/lib-2026-10-15-small-language-model-serving-for-search-and/"
original_url: "https://tech.meituan.com/2026/10/15/Small-Language-Model-in-meituan.html"
tags: ["lane a", "ai technology", "inference cost"]
---

# Small language model serving for search and recommendation

Original title: 搜推场景小语言模型在线推理工程实践

## English summary

Production SLM serving under 20 ms mean / 70 ms P99 (80 ms SLA): prefill-only MFU 67%, about 10x SGLang throughput; selective FP8 +24%, shared-prefix attention +20%; peak throughput is not SLA capacity (more concurrency gave a few % QPS for +30% latency).

Original: https://tech.meituan.com/2026/10/15/Small-Language-Model-in-meituan.html