--- id: "lib-2026-11-02-which-jobs-can-a-local-llm-do-a-slack-bot-on-dgx" type: "library" title: "Which jobs can a local LLM do? A Slack bot on DGX Spark" title_original: "ローカルLLMはどんな業務で使えるのか？ DGX Spark + DeepSeek V4-FlashのSlackチャットボットを評価した" lang: "JA" org: "Sompo Japan DX (Zenn)" source_type: "Practitioner post" date: "2026-11-02" week: "2026-W44" url: "https://nosov.ai/library/lib-2026-11-02-which-jobs-can-a-local-llm-do-a-slack-bot-on-dgx/" original_url: "https://zenn.dev/sompojapan_dx/articles/05f34316118943" tier: "B" tier_name: "Useful, limited" tier_reason: "Sompo Japan engineer's evaluation of a Slack bot on one DGX Spark running DeepSeek V4-Flash, with a test set, latency figures and a clear pass/fail split; useful and specific, but a test environment, one quantised model and no cloud comparison." summary_basis: "full-text" original_minutes: "10" tags: ["ai technology", "local llm", "on prem inference", "task fit"] --- # Which jobs can a local LLM do? A Slack bot on DGX Spark Original title: ローカルLLMはどんな業務で使えるのか？ DGX Spark + DeepSeek V4-FlashのSlackチャットボットを評価した Tier B (Useful, limited): Sompo Japan engineer's evaluation of a Slack bot on one DGX Spark running DeepSeek V4-Flash, with a test set, latency figures and a clear pass/fail split; useful and specific, but a test environment, one quantised model and no cloud comparison. Summary based on the full text. ## In short A Zenn post from Sompo Digital Lab (2 November 2026) by an engineer at Sompo Japan building AI agent systems. To keep internal conversations and documents off external APIs, he built a Slack bot whose model runs on a single NVIDIA DGX Spark: DeepSeek V4-Flash, quantised to about 104 GB, served by llama-server. Research Q&A across Slack, Notion and GitHub passed if users can wait 10 seconds to 5 minutes; code generation failed and is handed to Claude Code after human approval. Latency came mostly from reading long prompts, and fixing the prompt prefix so caching works halved response time. All writes go through an approval button. ## Key numbers - about 300 / about 30 tokens per second: prompt reading and text generation speed on one DGX Spark with a 3-bit-class quantised model - about 5 to about 2 minutes: answer time for the same question after shrinking the prompt from about 40,000 to about 6,000 tokens and fixing its prefix - 6/8 and 5/8: code-generation tasks with correct output in two trials; code generation judged a fail - 44: code generations run (11 tasks under 4 conditions) - about 2 weeks: one person's time to build the first test version ## What it reports - Hardware: one DGX Spark with 128 GB shared memory. Running costs are electricity and Claude Code fees; no per-token billing. The model has 284B total and 13B active parameters per token (sparse MoE) and is a reasoning model. - One conversation at a time; with more than 5 queued, the bot replies "busy" at once so users do not assume it froze. - Research Q&A: simple questions about 10 seconds, Notion lookups about 2 minutes, multi-tool research 2-5 minutes. It counts as usable because the bot reacts immediately, shows its plan as a checklist and ticks items off. - Code generation (JavaScript scripts): correct output in 6 and 5 of 8 tasks over two trials, dependencies reproduced in 2 of 3 tasks, and one use of a banned identifier. - Writes to Notion, GitHub issues, reminders and long-term memory, including memory clean-up and deletion, need a human click in Slack; tools are allow-listed; the bot reads only channels it is invited to and pre-approved Figma files. Claude Code runs in GitHub Actions from an approved issue; humans review and merge, and the bot only notifies on done, waiting for input, or error. - Latency causes: a large Notion tool schema sent with every question, the current time down to the minute in the system prompt (breaking the cache every request), and large tool results re-read each step. Moving the timestamp next to the question and slimming the schema took the prompt from about 40,000 to about 6,000 tokens. - Inserting instructions mid-history dropped cache hits to zero even with a matching prefix, because of how llama.cpp checkpoints this model's sparse attention. - Empty answers occurred when reasoning used up max_tokens; fixes were a higher limit, a fallback message, and one extra line at the end of the prompt on retry. Thousands of characters of repeated output also broke tool-call JSON, possibly from the heavy quantisation. - Two off-the-shelf agent products were rejected: one assumes a single user, the other has a self-modifying learning loop that does not fit internal controls. ## How they got it A test prompt set with defined expected behaviour, each run at least twice against the live model, in a test workspace with no business data. ## Limits - No comparison with cloud models under the same conditions, as the author states. - One model at 3-bit-class quantisation; other local models were not tested. - Test environment only; production use is out of scope. - Small code-generation test set. ## Why it matters for companies For firms that want internal data to stay on premises, this gives a realistic picture of a single-box setup today: fine for research Q&A with patient users, not for coding. The broader lessons apply to any agent deployment: put every write behind an approval step to close prompt-injection exits, keep system prompts static so caching works, and build a test set first so the line between local, cloud and human work can be argued with numbers. ## Key passages in translation > 応答遅延の主因は文章生成ではなく、入力プロンプトの読み込みでした。 The main cause of slow responses was reading the input prompt, not generating text. > ユーザー体験を損なう本質は「待ち時間の長さ」そのものよりも「今動いているのか止まっているのか分からない不安」にあると考え We judged that what really hurts the user experience is less the length of the wait than not knowing whether it is working or stuck. > 「AIにどこまで権限を与えてよいか」という社内議論を、「どの承認ボタンを用意するか」という具体的な論点に落とし込めます。 It turns the internal debate over how much authority to give AI into the concrete question of which approval buttons to provide. Original: https://zenn.dev/sompojapan_dx/articles/05f34316118943