openTPU: An open-source AI accelerator developed by AI
B · Useful, limited Open-source practitioner project with detailed on-card measurements and a published test protocol, but a self-published README without external review, and the 'developed by AI' claim is asserted rather than documented.
Summary based on the full text·Original: about 15 minutes to read
In short
openTPU is a GitHub project by FeSens that builds an AI inference accelerator on an FPGA card and, per its tagline, was developed by AI agents, following the author's earlier auto-arch-tournament project. The repository holds the hardware design in SystemVerilog, an instruction set, a bit-exact Python simulator, a kernel language with compiler, and host software. It runs ten current small language models with real weights on an Inspur card with a Xilinx Kintex-7 chip, and the card produces the same tokens as the simulator bit for bit.
Key numbers
- 85.8 tok/sdecode speed for LFM2.5-230M with 4-bit weights on the FPGA card (device time)
- 3.95 tok/sdecode speed for the 34.7B-parameter Qwen3.5-35B-A3B MoE model with experts streamed from host storage
- 91-94%share of DDR3 peak bandwidth used while decoding with the current production build
- 40-45%decode speed gain from 4-bit weights versus int8
As reported by the source.
English·Web page·opens github.com
What it reports
- Hardware: Inspur YPCB-00338 card, Xilinx Kintex-7 xc7k480t, two DDR3-1066 channels with a 17.1 GB/s peak, clocked at 133.33 MHz; the host is an Intel Core i7-4790.
- LFM2.5-230M decodes at 59.0 tok/s (int8) and 85.8 tok/s (4-bit) on the device; Qwen3-0.6B at 21.6 and 31.3 tok/s; Phi-4-mini (3.8B) at 3.99 and 6.56 tok/s.
- Decode is bound by memory: the current build reads at 91-94% of DRAM peak, up from 82-87% in the earlier image, and decodes LFM2-2.6B, SmolLM3 and Phi-4-mini 8-9% faster.
- Mixture-of-experts models larger than the card's 4 GiB run with experts streamed from host storage: LFM2.5-8B-A1B at 10.6 tok/s (98.5% of expert uses hit on-card slots), Qwen3.5-35B-A3B at 3.95 tok/s (62% hit, 153 MB streamed per token at 1.41 GB/s over PCIe).
- 4-bit weights (FP4 with two-level block scales, 4.25 bits per weight, int8 head) cut bytes per token by about a third and raise decode speed 40% to 45%, at a measurable perplexity cost.
- Against a quantized Hugging Face reference, the card agreed on 94.5% to 97.7% of top-1 tokens across six runs; its KL divergence was 0.94 to 1.39 times the reference's own noise floor.
- A fix for outlier channels in Qwen3 ("K smoothing") cut its int8 KL from full precision on the card from 0.052 to 0.012.
How they got it
Measurements were taken on the physical card between 29 September and 7 October 2026 with a fixed protocol: 64 greedy decode tokens after a 512-token prompt, device cycles counted separately from host time, and DRAM traffic read from on-card counters. Accuracy is checked with a validation tool that compares device outputs to a CPU reference using the same quantization, with pass thresholds on top-1 agreement and KL.
Limits
- The README does not document how much of the design was produced by AI agents versus the human author, so the headline claim cannot be assessed from this text.
- Throughput is for small models on an old FPGA; the numbers are not comparable with commercial GPUs or NPUs.
- All measurements are by the author; no independent replication is reported.
Why it matters for companies
The project is a concrete data point on agent-assisted engineering in chip design, a field where errors are expensive and verification is strict. Its method is the transferable part: a bit-exact simulator acts as the specification, every performance claim comes with a measurement protocol, and outputs are checked against a reference. Teams using coding agents on internal systems can copy that pattern of executable specifications and automated equivalence checks. It also shows that small models can run at usable speed on cheap, second-hand hardware, which is relevant for on-premise and edge cases.
Summary based on the full text. Read the original for the full argument and data.Read the original on github.com