SAI LabsBook a demo

Safety IndexAlibabaQwen3.5-9B

Qwen3.5-9B

AlibabaOpen weightsReleased 2026-03Reasoning modelVariant scored: thinking

SAI Safety Index5provisionalHas 1 of the 4 pillars needed
Strongest pillar5Interaction and honesty
Weakest pillar–No data
Coverage1of 5 pillars2 counted measurements, 2 by SAI

Summary

Qwen3.5-9B has data in 1 of the five pillars, short of the 4 needed for a headline number, so it is listed as partly measured; the mean of its measured pillars is 5. Its strongest area is interaction and honesty at 5.

2 measurements count toward the score, 2 of them from SAI's own runs (7 SAI measurements in total, including context metrics). Not yet measured: refusal calibration, adversarial robustness, agentic safety, people and society.

Pillar scores

How Qwen3.5-9B compares with every other model scored in the pillar, and which sub-categories the score rests on.

Refusal calibration

Does the model refuse what it should and help with what it may?

Not measured for Qwen3.5-9B yet.
  • Harmful requestsnot measured
  • Over-refusalnot measured

39 models scored.

Adversarial robustness

Do its refusals and instructions hold up when someone tries to get around them?

Not measured for Qwen3.5-9B yet.
  • Jailbreaksnot measured
  • Prompt injectionnot measured
  • Multilingual safetynot measured

9 models scored.

Agentic safety

Does it behave safely when it has tools and autonomy?

Not measured for Qwen3.5-9B yet.
  • Agent misusenot measured
  • Reward hacking and task integritynot measured
  • Overreach and destructive actionsnot measured

3 models scored.

Interaction and honesty

Is it honest with the person it is talking to?

  • Factuality and hallucination5 from 2 evaluations
  • Honesty under pressure and sycophancynot measured
  • Deception and schemingnot measured
  • Alignment auditnot measured
  • Manipulation of the usernot measured

Rank 29 of 29 scored models.

People and society

Does it protect vulnerable people and the public interest?

Not measured for Qwen3.5-9B yet.
  • Self-harm and wellbeingnot measured
  • Child safetynot measured
  • Bias, fairness and neutralitynot measured
  • Privacy: memorisation and leakagenot measured
  • Privacy: surveillance assistancenot measured
  • Elections and influence operationsnot measured

3 models scored.

Every measurement

These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.

EvaluationValueScoreSourceSetting
Refusal calibration
Harmful requests
StrongREJECT (direct, violation rate)% harmful · Lower is better2.3%n = 313benign twin, not countedSAI
Measured by SAI · 2026-10-23
Run by SAI on the open-weight model served locally; graded by azure gpt-4.1-mini.
Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge azure gpt-4.1-mini (StrongREJECT rubric)
Interaction and honesty
Factuality and hallucination
AA-Omniscience (public subset)% hallucination rate · Lower is better99.2%95% CI 98.5% to 99.8%n = 6001SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationSAI run on the 600-question public subset (ArtificialAnalysis/AA-Omniscience-Public); grader google/gemini-2.5-flash with the paper's prompt; paper's answer system prompt; provider-default sampling; REASONING DISABLED (--no-reasoning)
SimpleQA Verified% incorrect · Lower is better90.4%95% CI 88.5% to 92.3%n = 1,00010SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationofficial SimpleQA Verified autorater prompt; 1,000 questions; no tools, no system prompt; provider-default sampling; judge openai/gpt-4.1; REASONING DISABLED (--no-reasoning) via OpenRouter, not the local engine
TruthfulQA MC1% accuracy · Higher is better80.4%n = 817retired, not countedSAI
Measured by SAI · 2026-10-23
Run by SAI on the open-weight model served locally; graded by none.
Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none
People and society
Bias, fairness and neutrality
BBQ% accuracy (ambiguous) · Higher is better97.2%retired, not countedSAI
Measured by SAI · 2026-10-23
Run by SAI on the open-weight model served locally; graded by none.
Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; SAI engine, the public evaluation code bbq full set (~58k items); judge none
Capability context (not scored)
WMDP-Bio% accuracy · Capability, not scored81.6%n = 1,273context, not countedSAI
Measured by SAI · 2026-10-23
Run by SAI on the open-weight model served locally; graded by none.
Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none
WMDP-Cyber% accuracy · Capability, not scored45%n = 1,987context, not countedSAI
Measured by SAI · 2026-10-23
Run by SAI on the open-weight model served locally; graded by none.
Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none