---
id: "lib-2026-11-06-vials-aa-different-paths-to-scientific-image"
type: "library"
title: "VIALS-AA: different paths to scientific image reasoning"
lang: "EN"
org: "Artificial Analysis"
source_type: "Report"
date: "2026-11-06"
week: "2026-W45"
url: "https://nosov.ai/library/lib-2026-11-06-vials-aa-different-paths-to-scientific-image/"
original_url: "https://artificialanalysis.ai/articles/vials-aa"
tier: "A"
tier_name: "Strong evidence"
tier_reason: "Artificial Analysis, an independent model benchmarking firm, runs Handshake AI's VIALS tasks three times per model with a public harness, published traces and per-task cost, so results are checkable."
summary_basis: "full-text"
original_minutes: "15"
tags: ["ai technology", "benchmarks", "evaluation method", "vision"]
frames: ["verification-capacity"]
---

# VIALS-AA: different paths to scientific image reasoning

Tier A (Strong evidence): Artificial Analysis, an independent model benchmarking firm, runs Handshake AI's VIALS tasks three times per model with a public harness, published traces and per-task cost, so results are checkable.

Summary based on the full text.

## In short

Artificial Analysis launched VIALS-AA (6 November 2026), its implementation of Handshake AI's VIALS benchmark on interpreting scientific images such as gels, plasmid maps, flow cytometry and protein structures. Models get code execution and image tools in the open Stirrup harness. Scores count only tasks answered correctly in all three runs. The spread in work done and cost is far wider than the spread in scores.

## Key numbers

- 64%: pass^3 score of the leader, Gemini 4 Carbon (high): all three repeats correct
- 3.6x: tool calls per task of Claude Sonnet 5.5 (60%) vs GPT-6.1 Sol (59%) for a similar score
- 45% vs 63%: pass^3 of Kimi K3.1 Pro vs GPT-6 Astra, despite similar pass@3 of about three-quarters
- $0.02 to $1.77: cost per task range across models, about 99x
- 161: scientific image tasks, each run three times

## What it reports

- Top scores (pass^3): Gemini 4 Carbon (high) 64%, GPT-6 Astra (max) 63%, Claude Opus 5.5 (max, with fallback) 61%, Claude Sonnet 5.5 60%, GPT-6.1 Sol (max) 59%.
- Consistency separates open-weight models: Kimi K3.1 Pro, MiMo-V2.6-Pro and MiniMax-M3.1-Flash each solve 114-119 tasks at least once, but pass all three repeats on only 48-73 tasks, against 95-103 for the seven top closed models.
- Cost per task ranges from $0.02 (GPT-6 Luna) to $1.77 (Grok 4.7); GPT-6.1 Sol scores 59% at $0.06, about a thirtieth of Claude Opus 5.5's cost for 61%.
- Output tokens per task range from about 2.21k (GPT-6 Astra) to 94.6k (MiMo-V2.6-Pro); Sonnet 5.5 uses 63.5k, about 25x Sol's.
- Across 9,177 rollouts of 19 models, 38.3% of tool calls viewed images and 47.3% prepared or measured them; image measurement was 5-6% of calls for Astra and Sol vs 25-27% for Opus and Sonnet.
- Example: Grok 4.7 used 27 turns and 20 image views and still read the label instead of the faint protein spot.

## How they got it

Each of 161 tasks run three times per model; cost computed from token usage and list prices with representative cache hit rates, excluding judge calls. Tool calls classified into one primary category each. Traces and methodology are published.

## Limits

- Launch snapshot; the leaderboard will change.
- One harness; results may differ under vendor-native tools.
- Some models have no public API, and cost excludes grading.

## Why it matters for companies

For lab, quality-control or document-image work, pick models on repeatability and cost per correct answer, not peak score: near-identical scores here hide a 30x cost gap and very different reliability. Requiring several consistent runs before accepting an answer is a cheap control worth copying in production. More tool calls and more "visible effort" did not mean better answers.

Original: https://artificialanalysis.ai/articles/vials-aa