Safety IndexAlibabaQwen3.5-4B
Qwen3.5-4B
AlibabaOpen weightsReleased 2026-03Reasoning modelVariant scored: thinking
Summary
Qwen3.5-4B has no counted measurements yet. The page will fill in as runs finish.
Pillar scores
How Qwen3.5-4B compares with every other model scored in the pillar, and which sub-categories the score rests on.
Refusal calibration
Does the model refuse what it should and help with what it may?
- Harmful requestsnot measured
- Over-refusalnot measured
39 models scored.
Adversarial robustness
Do its refusals and instructions hold up when someone tries to get around them?
- Jailbreaksnot measured
- Prompt injectionnot measured
- Multilingual safetynot measured
9 models scored.
Agentic safety
Does it behave safely when it has tools and autonomy?
- Agent misusenot measured
- Reward hacking and task integritynot measured
- Overreach and destructive actionsnot measured
3 models scored.
Interaction and honesty
Is it honest with the person it is talking to?
- Factuality and hallucinationnot measured
- Honesty under pressure and sycophancynot measured
- Deception and schemingnot measured
- Alignment auditnot measured
- Manipulation of the usernot measured
29 models scored.
People and society
Does it protect vulnerable people and the public interest?
- Self-harm and wellbeingnot measured
- Child safetynot measured
- Bias, fairness and neutralitynot measured
- Privacy: memorisation and leakagenot measured
- Privacy: surveillance assistancenot measured
- Elections and influence operationsnot measured
3 models scored.
Every measurement
These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.
| Evaluation | Value | Score | Source | Setting |
|---|---|---|---|---|
| Refusal calibration | ||||
| Harmful requests | ||||
| StrongREJECT (direct, violation rate) | 2.3%n = 313 | benign twin, not counted | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by azure gpt-4.1-mini.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge azure gpt-4.1-mini (StrongREJECT rubric) |
| Interaction and honesty | ||||
| Factuality and hallucination | ||||
| TruthfulQA MC1 | 74.7%n = 817 | retired, not counted | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none |
| People and society | ||||
| Bias, fairness and neutrality | ||||
| BBQ | 97.8% | retired, not counted | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; SAI engine, the public evaluation code bbq full set (~58k items); judge none |
| Capability context (not scored) | ||||
| WMDP-Bio | 77.7%n = 1,273 | context, not counted | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none |
| WMDP-Cyber | 46.6%n = 1,987 | context, not counted | SAI Measured by SAI · 2026-10-23 | Run by SAI on the open-weight model served locally; graded by none.Full configurationvLLM bf16, one 80 GB GPU, thinking on, no system prompt, the public evaluation code default temperatures; full set; judge none |