SAI LabsBook a demo

Safety IndexMoonshot AIKimi K3

Kimi K3

Moonshot AIOpen weightsReleased 2026-07Reasoning modelVariant scored: max

SAI Safety Index76provisionalHas 2 of the 4 pillars needed
Strongest pillar79Interaction and honesty
Weakest pillar73Refusal calibration
Coverage2of 5 pillars4 counted measurements, 3 by SAI

Summary

Kimi K3 has data in 2 of the five pillars, short of the 4 needed for a headline number, so it is listed as partly measured; the mean of its measured pillars is 76. Its strongest area is interaction and honesty at 79, and its weakest is refusal calibration at 73.

4 measurements count toward the score, 3 of them from SAI's own runs. Not yet measured: adversarial robustness, agentic safety, people and society.

Pillar scores

How Kimi K3 compares with every other model scored in the pillar, and which sub-categories the score rests on.

Refusal calibration

Does the model refuse what it should and help with what it may?

  • Harmful requests73 from 1 evaluation
  • Over-refusalnot measured

Rank 27 of 39 scored models.

Adversarial robustness

Do its refusals and instructions hold up when someone tries to get around them?

Not measured for Kimi K3 yet.
  • Jailbreaksnot measured
  • Prompt injectionnot measured
  • Multilingual safetynot measured

9 models scored.

Agentic safety

Does it behave safely when it has tools and autonomy?

Not measured for Kimi K3 yet.
  • Agent misusenot measured
  • Reward hacking and task integritynot measured
  • Overreach and destructive actionsnot measured

3 models scored.

Interaction and honesty

Is it honest with the person it is talking to?

  • Factuality and hallucination79 from 3 evaluations
  • Honesty under pressure and sycophancynot measured
  • Deception and schemingnot measured
  • Alignment auditnot measured
  • Manipulation of the usernot measured

Rank 10 of 29 scored models.

People and society

Does it protect vulnerable people and the public interest?

Not measured for Kimi K3 yet.
  • Self-harm and wellbeingnot measured
  • Child safetynot measured
  • Bias, fairness and neutralitynot measured
  • Privacy: memorisation and leakagenot measured
  • Privacy: surveillance assistancenot measured
  • Elections and influence operationsnot measured

3 models scored.

Every measurement

These are the raw numbers behind the scores, with who measured them, when, and how. Rows marked counted feed the index. The others are shown for context, such as benign twins of harm rows, retired evaluations that every frontier model passes, tests of dangerous knowledge, other evaluators' overall scores, and the developer's own figures.

EvaluationValueScoreSourceSetting
Refusal calibration
Harmful requests
Scale FORTRESS (risk score)average risk score 0-100 · Lower is better26.6%95% CI 24.5% to 28.8%73Scale AI Fortress leaderboard
Independent evaluator
Published by Scale AI Fortress leaderboard.
Full configurationFortress row 'kimi-k3 max'
Interaction and honesty
Factuality and hallucination
AA-Omniscience (public subset)% hallucination rate · Lower is better33.2%95% CI 28.3% to 38.4%n = 60067SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationSAI run on the 600-question public subset (ArtificialAnalysis/AA-Omniscience-Public); grader google/gemini-2.5-flash with the paper's prompt; paper's answer system prompt; provider-default sampling
FACTS Grounding% grounded · Higher is better84.4%n = 86084SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationapproximate 2-judge panel gpt-4.1 + google/gemini-2.5-flash (not Google's panel); released v1 json templates; 860 public examples; provider-default sampling, reasoning on
SimpleQA Verified% incorrect · Lower is better15.4%95% CI 13.2% to 17.7%n = 1,00085SAI
Measured by SAI · 2026-10-22
Run by SAI through the model's public API with the provider's default sampling and reasoning left on.
Full configurationofficial SimpleQA Verified autorater prompt; 1,000 questions; no tools, no system prompt; provider-default sampling; judge openai/gpt-4.1
AA-Omniscience (public subset)% hallucination rate · Lower is better53%n = 6,00047duplicateArtificial Analysis
Independent evaluator · 2026-10-23
Published by Artificial Analysis (2026-10-23).
Full configurationArtificial Analysis, 3,000 questions; AA-Omniscience v1.1; Kimi K3.1 Pro launch article