---
id: "lib-2026-10-12-having-chinese-open-weight-models-hunt"
type: "library"
title: "Having Chinese open-weight models hunt vulnerabilities in our own service"
title_original: "中国製オープンウェイトモデルに、自社サービスの脆弱性を探させた"
lang: "JA"
org: "Neurogica (Zenn)"
source_type: "Practitioner post"
date: "2026-10-12"
week: "2026-W42"
url: "https://nosov.ai/library/lib-2026-10-12-having-chinese-open-weight-models-hunt/"
original_url: "https://zenn.dev/neurogica/articles/ef9ebaa4f3e9bd"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "Small Japanese AI firm's engineer reports a controlled pre-release test of four open-weight models on its own service, with the setup and every finding classified; the method is clear but the sample is eight runs on one codebase."
summary_basis: "full-text"
original_minutes: "7"
tags: ["ai security", "vulnerability discovery", "open weight models"]
---

# Having Chinese open-weight models hunt vulnerabilities in our own service

Original title: 中国製オープンウェイトモデルに、自社サービスの脆弱性を探させた

Tier B (Useful, limited): Small Japanese AI firm's engineer reports a controlled pre-release test of four open-weight models on its own service, with the setup and every finding classified; the method is clear but the sample is eight runs on one codebase.

Summary based on the full text.

## In short

A Zenn post (12 October 2026) by Shunya Nagashima of Neurogica, written before the launch of the company's time-series forecasting service. Prompted by a JPCERT/CC alert on a run of Japanese data leaks, the team ran four Chinese-origin open-weight models (Altar-1, Qwen3.8-Flash-Next, GLM-5.3, DeepSeek V4 Pro 0813) against its own code to look for authentication, authorisation and billing flaws. The case for open weights: closed models restrict cyber use, attackers can download open models anyway, and running locally keeps source code and unfixed bugs off external inference services.

## Key numbers

- 16: candidate vulnerabilities reported across the four models
- 2: candidates confirmed, reproduced and fixed (one from Qwen, one from GLM-5.3)
- 10: candidates judged false positives; 4 more still unverified
- 2 of 8: runs that ended without a complete report
- 1,440GB: total VRAM of the eight B200 GPUs used to run each model locally

## What it reports

- Same code, prompt and tools for each model; two runs each, up to 60 minutes and 30 model calls per run, with no carry-over between runs.
- 16 candidates in total: 2 confirmed, 10 false positives, 4 unverified. Qwen gave 8 candidates (5 false), GLM 4 (3 false).
- Qwen's confirmed finding (Medium): client identity was lost through MCP, so a rate limit on a failed-auth client also hit unrelated users.
- GLM's confirmed finding (Low): a payment amount or currency mismatch was rejected, but the dedicated audit log was never written.
- Altar-1 and DeepSeek produced no confirmed findings. One DeepSeek run sent zero HTTP requests and reported nothing; one Altar-1 run hit the output limit mid-report.
- Each model found its real issue in only one of its two runs.

## How they got it

Local inference on eight NVIDIA B200 GPUs, one model at a time. Every candidate was triaged by people; the two confirmed issues were reproduced and re-tested after the fix. The post lists the published GPU configuration each model is reported to run on, and notes Qwen was run in BF16 so a quantised single-GPU version is untested.

## Limits

- Eight runs on one service; the author says more repeats are needed to judge whether results are stable.
- Duplicate findings were merged per model, so the 16 candidates are not all distinct issues.
- No comparison with a closed model, a conventional scanner or a human pentest on the same code.
- The claim that recent Japanese attacks use AI is the author's belief, stated as such.

## Why it matters for companies

The yield was 2 real issues from 16 candidates, with human reproduction still required, which is a realistic baseline for AI-assisted security review. Running open weights in-house is a way to scan code and known bugs without sending them to a provider, and to fix model and procedure so scans repeat under the same conditions. The suggested cadence is concrete: rescan on changes to auth or billing, before releases, and when new models appear, and turn each confirmed finding into a test.

## Key passages in translation

> 4モデルが報告した候補は延べ16件。そのうち確認済みが2件、誤検出が10件、まだ判定できていない候補が4件だった。

The four models reported 16 candidates in total. Of these, 2 were confirmed, 10 were false positives and 4 could not yet be judged.

> 同じサービスでも毎回拾えるわけではないので、安定して見つけられるかは、もう少し回数を増やして確かめたい。

Even on the same service they do not catch it every time, so we want more runs to check whether they find it reliably.

> AIでコードを書く速度を上げるなら、確かめる頻度も上げる。

If you raise the speed of writing code with AI, raise the frequency of checking it too.

Original: https://zenn.dev/neurogica/articles/ef9ebaa4f3e9bd