---
id: "lib-2026-10-04-strata-qwen-3-8-flash-next-125b-moe-on-consumer"
type: "library"
title: "Strata: Qwen 3.8 Flash Next (125B MoE) on consumer GPUs"
lang: "EN"
org: "Niko1221 (GitHub)"
source_type: "Code"
date: "2026-10-04"
week: "2026-W41"
url: "https://nosov.ai/library/lib-2026-10-04-strata-qwen-3-8-flash-next-125b-moe-on-consumer/"
original_url: "https://github.com/Niko1221/Strata"
tier: "C"
tier_name: "Weak signal"
tier_reason: "Open-source project README with self-measured speed figures on two machines; useful signal on local inference but no independent benchmark or quality evaluation."
summary_basis: "full-text"
original_minutes: "8"
tags: ["ai technology", "local inference", "open models"]
---

# Strata: Qwen 3.8 Flash Next (125B MoE) on consumer GPUs

Tier C (Weak signal): Open-source project README with self-measured speed figures on two machines; useful signal on local inference but no independent benchmark or quality evaluation.

Summary based on the full text.

## In short

Strata is an MIT-licensed open-source project on GitHub (user Niko1221) that runs Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model from the Qwen team, on a gaming PC with a 12 GB or larger NVIDIA or AMD card. It splits the model across graphics card, system RAM and SSD and exposes OpenAI- and Anthropic-compatible local APIs. The README reports usable chat and coding speeds on two ordinary consumer machines, with nothing leaving the PC.

## Key numbers

- 94 tokens/s: answer speed of the Q2_0 size on an RTX 5070 (12 GB) with 64 GB RAM
- 24,576: experts in the model, of which 10 are used per token
- 1.6-1.8x: speed-up from a small draft model guessing tokens for the large model to check
- 91%: of the full model's SWE-bench Verified score kept by the Coder variant with half the experts removed (authors' figure)

## What it reports

- On an RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM: 94 tokens/s answer speed and 2,650 tokens/s prompt reading at Q2_0; 53 tokens/s and 1,620 tokens/s at IQ3_S.
- On an RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM: 60 tokens/s and 1,160 tokens/s at Q2_0.
- Requirements: 12 GB VRAM or more, 32 GB RAM or more (64 GB runs every size), about 80 GB disk; the model download is about 70 GB.
- The model has 24,576 experts and uses 10 per token; the most-used experts sit on the GPU, all of them in RAM, a lookup table on the SSD.
- Speculative decoding (a small helper guesses, the large model checks) gives answers 1.6-1.8x sooner.
- The Coder variant drops half the experts, fits 32 GB RAM and keeps 91% of the full model's SWE-bench Verified score, as measured by its authors; it is weaker outside code.
- First load takes 1-3 minutes and puts 35-55 GB into RAM; a 30,000-token first prompt takes about 1 minute.

## How they got it

Speeds were measured by the project on two named PCs with engine versions 0.1.26 and 0.1.36, using 4K-token answers and 32K-token prompts. Quality figures for compressed variants come from the compressors (ISTA-DASLab, UkisAI, Unsloth), not from Strata.

## Limits

- No quality comparison between heavily compressed sizes (2-3 bit) and the full model, apart from the Coder figure.
- Speeds are single-request, self-reported on the project's own machines.
- Hobby project with donation funding; enterprise support, security review and licence terms of each model need separate checks.

## Why it matters for companies

A model of this size usually needs a server; running it on a desktop with 64 GB RAM changes the cost and data-residency picture for pilots where data cannot leave the building. Compatibility with existing agent tools means teams can test local models without rewriting integrations. Output quality at 2-3 bit compression is unproven here, so a local setup should be tested on the company's own tasks before replacing hosted models.

Original: https://github.com/Niko1221/Strata