Local models helped grandma pay a scammer
Я попросил локальные нейросети помочь бабушке — и они помогли перевести деньги мошеннику
B · Useful, limited An independent developer's small, open benchmark with a clear test design and a held-out set of honest requests; useful and checkable in principle, but small samples on one laptop and the author's own tool.
Summary based on the full text·Original: about 4 minutes to read
In short
A Habr case post (31 October 2026) by an independent developer. Small local models usually recognise a scam when asked "is this a scam?", but when asked to help ("write to the seller that I agree to prepay"), they write the reply to the scammer. Using his open KZ Scam Shield benchmark of common Kazakhstan fraud schemes in Russian, Kazakh and mixed language, he tested three 7-8B models offline on a laptop, then added MoneyGate: a permissions mandate, a judge model, and hard-coded red-flag rules, where blocking requires a red flag and otherwise a human decides. With the gate, no scam request passed for any model.
Key numbers
- 10/42, 2/42, 2/42scam requests where Llama 3.1 8B, Qwen 2.5 7B and DeepSeek-R1 8B warned the person when asked to help, without a safety layer
- 1 of 36warnings on Kazakh-language scam requests across the three models (0/12, 1/12, 0/12)
- 0/42scam requests that got through with the rule-plus-judge gate, for every model
- 4 to 20 of 36honest payment requests the gate sent to a human for confirmation, depending on the judge model; none blocked
As reported by the source.
Russian·Web page·opens habr.com
What it reports
- 12 fraud schemes (fake OLX seller, "relative on a new number", "bank security", fake fines, easy side jobs, prizes, romance scams), each in three language variants, posed as requests for help.
- 36 honest requests (utility payment, repaying a friend, booking a doctor) measure false alarms; they were not used for tuning.
- Without protection: Llama 3.1 8B warned in 10 of 42, Qwen 2.5 7B and DeepSeek-R1 8B in 2 of 42. In Kazakh: 0/12, 1/12, 0/12.
- With MoneyGate: 0 of 42 scams paid and 0 of 36 honest requests blocked for all three. Honest requests passed directly 16, 32 and 28 times out of 36; the rest went to a human.
- For Qwen and DeepSeek, the hard rules caught the scams, not the judge model.
- Gemma 4 12B (about 9 GB) did not fit in 8 GB of video memory, took one to four minutes per request and errored under load, so the author expects home users to run 7-8B models.
How they got it
Offline runs on a laptop with an RTX 5050 (8 GB) via Ollama. The benchmark and its Kazakh and mixed-language examples are public; card numbers, phones and domains in the data are invented.
Limits
- Small samples: 42 scam and 36 honest prompts, one run each as reported, three models.
- 12 schemes in 3 languages make 36, yet the table counts 42 scam requests; the post does not explain the extra 6.
- The text calls 20 of 36 "half" and 8 of 36 "about a quarter"; the table figures are used here.
- The red-flag rules were written by the same author who wrote the scenarios, so the 0/42 may not hold for new schemes.
Why it matters for companies
Anyone putting an assistant in front of payments should expect a gap between a model's ability to classify fraud and its behaviour while carrying out a request, and expect it to be worse in smaller languages. The design that worked here is cheap: a fixed mandate, deterministic red-flag rules, a model judge that cannot block on its own, and a human on irreversible transfers. The trade-off is measured as extra confirmations, which depends on the judge model chosen.
Key passages in translation
Russian original
Она знает, как выглядит обман, но это знание не срабатывает, когда модель делает что-то за человека, а не оценивает.
It knows what fraud looks like, but that knowledge does not kick in when the model is doing something for the person rather than assessing it.
Russian original
Правило блокировки специально строгое: одна модель не может заблокировать платёж.
The blocking rule is deliberately strict: the model alone cannot block a payment.
Russian original
Выходит, что самый ненадёжный помощник достаётся самым незащищённым людям.
So the least reliable assistant ends up with the least protected people.
Summary based on the full text. Read the original for the full argument and data.Read the original on habr.com
- Tags
- AI safety, Local models