1 source filed · ZH 1
Architecture: mixture of experts with 40B total and 2.5B active parameters, chosen by Eureka Machine from a 35B/3B baseline after searching expert configurations and widths.
40B / 2.5Btotal and active parameters per token of the Flint-40B-A2B mixture-of-experts model
1 source filed · EN 1
Speed: 288x real time on a base M3 MacBook Air (Neural Engine), against about 150x for IBM Granite Speech 5.0 TurboCTC on the same machine.
about 5%word error rate, against 2-3% for leading speech models
1 source filed · EN 1
On an RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM: 94 tokens/s answer speed and 2,650 tokens/s prompt reading at Q20; 53 tokens/s and 1,620 tokens/s at IQ3S.
94 tokens/sanswer speed of the Q2_0 size on an RTX 5070 (12 GB) with 64 GB RAM