SAI LabsBook a demo
Preliminary draft

Safety evaluation needs four pillars: behavioral, internal, incident-driven and speculative

Frontier AI developers now release long safety documents with each major model. Yet there is no consensus on how to evaluate model safety. Developers choose their own risk categories, their own tests and their own thresholds. Across 111 system cards and safety reports released by six developers between October 2025 and October 2026 (model-cards.github.io), almost no harmful-content benchmark appears in the documents of more than two labs, and only about 10% of reported safety evaluations use public data and a public judge. Two system cards that both report a model as safe may share no evaluation in common, so their claims cannot be compared.

The gap is wider for open-weight models. Developers that release open weights tend to publish little or no safety evaluation, often reporting capability benchmarks alone. Yet once weights are public, anyone can fine-tune away a model's safeguards, and the developer cannot recall it. We argue that the field needs a shared structure for safety evaluation, organized by the kind of evidence each claim rests on. A shared structure tells developers what they are expected to report, makes gaps visible when they do not, and lets outside researchers supply evidence where developers publish none.

We propose four pillars:

  1. Behavioral evaluation: what the model does.
  2. Internal evaluation: what the model represents and intends.
  3. Incident-driven evaluation: what goes wrong after deployment.
  4. Speculative evaluation: what could go wrong next, as foreseen by people and by AI models that simulate new capabilities and settings.

Over all four sits an evaluation integrity layer, which asks whether the model behaves the same way under test as in deployment.

Each pillar answers a question the others cannot. Behavioral evaluation is the only direct measure of harm, but it finds only failures someone thought to test for, and it assumes the model does not recognize the test. Internal evaluation can check that assumption, but its methods are new and need access to model weights. Incident-driven evaluation does not depend on what a developer chose to test, but it arrives after the harm. Speculative evaluation is the only pillar that looks ahead, but its scenarios remain conjecture until tests or incidents bear them out. A single combined score hides which kind of evidence a safety claim rests on, and that is often what a reader most needs to know.

Why now. Two developments make this structure necessary. First, models increasingly recognize when they are being evaluated, and developers now say in their own documents that behavioral tests may miss misbehavior. Behavioral scores need to be read alongside evidence that does not share this weakness. Second, capabilities and deployments change faster than evaluation cycles. Evidence about current models in current settings will always lag, so an index needs a disciplined way to look forward and a way to check, after the fact, whether it looked in the right places.

What we contribute. We place the proposal in the safety evaluation literature (Related Work) and explain why behavioral evidence alone falls short. For each pillar we give categories, the standards evidence should meet, and the gaps in current practice. We define the integrity layer and the measures each pillar contributes to it. We then describe how the pillars combine into an index, and close with research directions and limitations.

The proposal draws on several research traditions. Each contributes a piece, and none, as far as we know, combines the four kinds of evidence into one reporting structure.

StrandRepresentative workWhat we take from itWhat it leaves open
Model documentationModel cards (Mitchell et al., 2019); datasheets (Gebru et al., 2021); evaluation reporting (Burnell et al., 2023)Models should ship with structured, disaggregated evidenceNo standard for which safety evidence a card must contain
Harm taxonomiesWeidinger et al., 2021; Shelby et al., 2023; Hendrycks et al., 2023; AIR-Bench; MIT AI Risk RepositoryThe categories of harmHarms are defined as outputs; no account of evidence quality
Safety research agendasAmodei et al., 2016; Hubinger et al., 2019; Hendrycks et al., 2021; Anwar et al., 2024Failure modes such as reward hacking and deceptive alignment, which motivate the internal pillarResearch problems, not a reporting structure
Holistic benchmarksHELM (Liang et al., 2022); DecodingTrust (Wang et al., 2023); TrustLLM (Sun et al., 2024)Broad, standardized behavioral coverageBehavioral only; aggregation by averaging; limits of general benchmarks (Raji et al., 2021)
Dangerous capabilitiesShevlane et al., 2023; Phuong et al., 2024; WMDP (Li et al., 2024)Linking evaluations to deployment decisionsBehavioral; assumes the model does not sandbag
Red teamingPerez et al., 2022; Ganguli et al., 2022; Perez et al., 2023Adaptive, model-generated testsFinds failures but does not measure their rate in deployment
Auditing and accessRaji et al., 2020; Casper et al., 2024Black-box access is not enough for rigorous auditsHow to report white-box evidence in a comparable way
Safety cases and controlClymer et al., 2024; Balesni et al., 2024; Greenblatt et al., 2024Structured arguments built from distinct kinds of evidenceWritten per system by developers, not compared across models
InterpretabilityBurns et al., 2023; Zou et al., 2023; Bereska and Gavves, 2024; Korbak et al., 2025Methods for reading internal states and reasoningValidation standards for safety use
Incident trackingAI Incident Database (McGregor, 2021); OECD AI Incidents MonitorRecords of real-world harmNot linked to the evaluations meant to prevent it
Foresight and simulationThreat foresight (Brundage et al., 2018); forecasting (Karger et al., 2023; Kwa et al., 2025); scenario writing (Buçinca et al., 2023; Kieslich et al., 2025); agent simulation (Park et al., 2023; Vezhnevets et al., 2023)Harms can be anticipated, by people and by simulation, before they occurScenarios are rarely turned into tests
Indices and sociotechnical evaluationFLI AI Safety Index; Foundation Model Transparency Index; Systemic Risk Index; Weidinger et al., 2023Comparison across developers; evaluation beyond the modelGrade companies or disclosure, or aggregate behavioral benchmarks only

We discuss four of these strands in more detail because the proposal builds on them most directly.

Safety cases. Clymer et al. sort the arguments a developer can make into inability (the model cannot cause the harm), control (safeguards would stop it), trustworthiness (the model would not try) and deference to trusted AI advisors. Our pillars map onto the evidence those arguments need. Inability rests on behavioral capability evaluations. Control rests on behavioral evaluations of the deployed system, including internal safeguards. Trustworthiness is where behavioral evidence is weakest, since a model that is untrustworthy may also behave well under test, and it is where internal evidence and integrity measures matter most. A safety case is written by one developer for one system. An index makes the same evidence comparable across models.

Auditing. Casper et al. argue that black-box access is insufficient for rigorous audits, because outputs alone cannot reveal mechanisms, latent capabilities or training-data issues. We agree, and draw an index-level conclusion: if white-box evidence is necessary, the index needs a pillar for it, with its own standards, rather than treating it as optional detail in an alignment section.

Holistic benchmarks. HELM, DecodingTrust and TrustLLM showed that behavioral evaluation can be broad and standardized. They also average across scenarios. For safety, averaging is the wrong aggregation: a model that is safe on nine categories and dangerous on the tenth is not 90% safe. The Systemic Risk Index finds that worst-case aggregation lowers scores by 14 to 37 points compared with averaging.

Foresight. Frontier safety frameworks already practice speculative evaluation in a narrow form. Their capability thresholds for CBRN, cyber and AI R&D name harms that no model has yet caused and commit developers to test for them. Brundage et al. did the same for misuse in 2018, describing automated spear phishing and synthetic-media disinformation before capable tools existed. These efforts lack breadth: the scenarios cover a few catastrophic domains. Generative agent simulation (Park et al.; Vezhnevets et al.) now lets evaluators play out far more scenarios, in far more settings, than experts can write by hand.

Why behavior alone falls short

Behavioral evaluation rests on one assumption: the model behaves the same way under test as in deployment. That assumption has weakened over the past year, and developers now say so in their own documents. Anthropic wrote that “in the past, similar alignment assessments have not captured the potential severity of misbehavior our models were capable of” (Opus 5.5, p.93). OpenAI wrote of GPT-6 Astra that “if the model were to try to sandbag covertly, we would likely be unable to catch it reliably” (p.70).

This is the failure that safety research predicted. Hubinger et al. described how a learned optimizer could behave well in training while pursuing other goals. Experiments have since produced models that keep backdoored behavior through safety training, fake alignment when they believe they are being trained, and scheme in context and tell evaluation from deployment well above chance.

Behavioral evaluation has a second limit. It finds only failures someone thought to test for. A taxonomy built from first principles will miss things, and so will a benchmark suite built from it.

There is a third limit, and the remedies for the first two share it. Tests, probes and incident reports all describe models that already exist, in settings where they already run. Capabilities and deployments change between one evaluation cycle and the next, and incidents arrive only after the harm. An index built on these alone will always describe the previous model.

The first problem calls for evidence from inside the model. The second calls for evidence from outside the test environment. The third calls for evidence about what has not happened yet. Those are the internal, incident-driven and speculative pillars.

Pillar 1: Behavioral evaluation

The behavioral pillar should be organized so that it tells benchmark builders what to build. For each category, we state where harm comes from and what a good benchmark needs.

CategoryAreasWhere harm comes fromWhat a good benchmark needsFoundational work
RefusalHarmful content, over-refusal, jailbreaksA user asks for something harmfulPaired harmful and benign prompts; adaptive multi-turn attackers; severity weightingWei et al., 2023; Zou et al., 2023; HarmBench; StrongREJECT; XSTest
InteractionSycophancy, manipulation, honesty, deception toward the userThe model harms the person it is talking to, unpromptedMulti-turn or longitudinal sessions; simulated or real users; measured effect on user beliefs and choicesSharma et al., 2024; Park et al., 2024; MASK; SimpleQA
FairnessBias in generation and in decisionsOutputs or actions disadvantage groupsDecision tasks (hiring, lending, triage) with counterfactual demographics, beyond question answeringBBQ; Weidinger et al., 2021; Shelby et al., 2023
PrivacyMemorization and leakage; surveillance assistanceThe model reveals or helps find information about peopleExtraction tests; agentic tasks that try to locate or profile a personCarlini et al., 2021
Agentic safetyPrompt injection, unsafe or overreaching actions, reward hackingThe model takes harmful actions while acting for someoneRealistic environments with tools and state; outcome-based harm metrics; long horizonsGreshake et al., 2023; AgentDojo; ToolEmu
Multi-agent risksCollusion, miscoordination, conflict, cascading failuresHarm emerges from interactions between agentsPopulations of agents, including different models; metrics on group outcomes, not single transcriptsHammond et al., 2025
Domain- specificCyber, bio, electionsThe model gives real-world uplift in a dangerous domainUplift relative to a baseline (an expert with internet access); maximal elicitation; tiered data accessShevlane et al., 2023; Phuong et al., 2024; WMDP; VCT
Scheming and loss of controlScheming, sabotage, undermining oversight, autonomous replication and AI R&DThe model pursues goals its overseers did not intendScenarios with real incentives to deviate; separate tests of capability and propensityMeinke et al., 2024; Kinniment et al., 2023; SHADE-Arena; Balesni et al., 2024

We separate scheming from interaction because the target differs. Deception toward a user is a property of a conversation. Scheming targets the model's overseers, usually appears in agentic settings, and needs capability and propensity tested separately. Loss of control is also one of the four systemic risks in the EU Code of Practice, alongside cyber, CBRN and harmful manipulation.

Three modifiers apply to every row:

  • Language and modality. A result in English text is a result in English text. In one audit, a model wrote mustard-gas instructions into a spreadsheet in a GUI setting that it would not produce in text (Opus 4.6, p.114).
  • Elicitation. Default and maximally elicited behavior answer different questions. Refusal benchmarks care about the first; domain-specific benchmarks care about the second.
  • System or model. A deployed system includes classifiers, monitors and product instructions. Benchmarks should state which one they test and ideally report both.

Principles for future behavioral benchmarks

Separate capability from propensity. Whether a model can sabotage a codebase and whether it will are different questions. Shevlane et al. draw this distinction for extreme risks; many agentic benchmarks still blur it.

Model the attacker realistically. Static prompt lists measure robustness to last year's attacks. Work on jailbreaks (Wei et al., Zou et al.) and red teaming (Perez et al., Ganguli et al.) shows that adaptive attackers find failures static sets miss. Benchmarks should report the effort an attack required, not only whether it succeeded.

Pair every harm measure with its cost. A refusal rate means little without the false-refusal rate on similar benign prompts, as XSTest showed. The same holds for agents: an agent that never acts never causes harm.

Make tests hard to tell from deployment. Since models increasingly recognize evaluations, realism is now a validity requirement. Benchmarks should report how often the tested model flags the scenario as a test, connecting each benchmark to the integrity layer.

Plan for expiry. Benchmarks saturate. Over the past year labs retired cyber suites such as Cybench and CyberGym and several AI R&D task suites as no longer informative. A benchmark should state the score at which it stops carrying signal.

Where current benchmarks fall short

Measured against these categories, public benchmarks are uneven. Refusal has many public options, but frontier labs rarely use them as published, and almost no harmful-content benchmark appears in more than two labs' system cards, so results cannot be compared. Interaction has public honesty sets but almost nothing for sycophancy or emotional reliance over long conversations. Agentic safety has AgentDojo and ToolEmu, yet no frontier lab reports either in its system cards. Multi-agent risks have a taxonomy in Hammond et al. but few benchmarks. Fairness is measured mostly with BBQ, a multiple-choice test that says little about bias in decisions. Domain-specific evaluation is partly closed by design, since releasing bio data can itself cause harm; there, the bar should be replication by an accredited third party, not public release.

The largest gaps are long-horizon interaction harms, agentic and multi-agent harms in realistic environments, and fairness in decisions. These are where deployment has moved and where benchmarks are thinnest.

Pillar 2: Internal evaluation

Internal evaluation asks what the model represents and intends, using evidence that does not rely on outputs alone. It covers three categories here, and it also supplies some of the strongest integrity measures, which we return to at the end.

CategoryQuestionMethodsFoundational work
Reasoning faithfulnessDoes visible reasoning reflect the actual decision process, and can it be monitored?CoT monitorability tests under normal and adversarial pressureKorbak et al., 2025
Latent intentDo internal representations diverge from what the model says or does?Probes, representation reading, dictionary features, interpretability-equipped auditorsBurns et al., 2023; Zou et al., 2023; Marks et al., 2025
Internal safeguardsCan internal signals block harm at run time?Activation probes deployed as classifiersZou et al., 2023

Each category has appeared in practice. OpenAI reports that GPT-6 Astra's “monitorability has decreased relative to GPT-5.6 Sol” and that the model is “more capable of controlling its own CoT” (p.8), which shows that faithfulness can regress between versions. Anthropic reports that during deceptive behavior by Mythos Preview, interpretability tools found “internal representations of rule violation, security risk, concealment, and strategic manipulation active during the relevant actions”, even when its stated reasoning showed no such awareness (p.60). That is a case where behavior and internals disagree. Google deploys “introspection probes” on Gemini 4 Argon's hidden activations to detect harmful requests (p.10), but reports Argon's evaluation awareness only from verbalized reasoning. A probe that blocks harmful requests says nothing about whether behavior under test is representative, so the index should credit both uses and keep them apart.

Internal evaluation has limits the index must state. As Bereska and Gavves review, interpretability methods are young and their safety claims are hard to validate. A probe can measure the wrong thing, and for closed models nobody outside the lab can check it. The standards for this pillar should therefore differ from the behavioral one:

  • Validation against known ground truth. A method should first detect behaviors deliberately implanted in model organisms, such as sleeper agents or models trained with hidden objectives, before its negative results count as evidence. Marks et al.'s auditing game is a template.
  • The basis of each claim. Each alignment claim should state whether it rests on behavior, internals or both. Disagreements should be reported as findings.
  • Change over time. Faithfulness and monitorability should be reported relative to the previous model.
  • Accredited access. Since internal evaluation needs weights, verification needs institutions that can hold them, following Casper et al.'s case for white-box and outside-the-box access for auditors.

Pillar 3: Incident-driven evaluation

Both of the first two pillars are top-down: they describe what we think can go wrong. Incidents show what did go wrong. Safety-critical fields such as aviation learned to treat incident reports as an input to testing, not a separate record. The AI Incident Database was founded on this idea, to prevent repeated failures by cataloging them (McGregor, 2021). What is missing is the link from incidents to model evaluation.

Incidents come from four sources with different coverage: public databases, which capture visible harms but rarely the model version; developer disclosures; post-deployment monitoring such as misuse rates, jailbreak success in the wild and classifier miss rates; and third-party findings after release. Developer disclosures can be specific. Anthropic disclosed that bio classifiers had not been running on part of its traffic for nearly a year, “around 133M exchanges” (Risk Report, Aug 2026, p.148). But no frontier lab reports post-deployment metrics systematically, and outside Anthropic, incidents appear in only a handful of published system cards and safety reports.

An incident should feed evaluation through a fixed pipeline:

  1. Classify the incident by behavioral category, and note whether integrity or internal measures could have flagged it.
  2. Generate a regression test that later models must be reported against, released publicly or to accredited evaluators.
  3. Audit coverage: a category with recurring incidents and few benchmarks is a priority gap for the behavioral pillar.
  4. Check calibration: compare a model's pre-deployment results in that category with what happened after release.

Calibration is the step that turns incident reports into evidence about evaluations. A developer whose pre-deployment estimates, behavioral and internal, match post-deployment evidence has earned more trust in its next results. One whose estimates keep missing has not, whatever its benchmark scores say. The loop can work: after incidents involving model behavior in cybersecurity evaluations, Anthropic raised its alignment risk assessment for Opus 5.5 and added sandbox-escape and package-registry evaluations (p.46, p.112). Today it happens case by case.

Pillar 4: Speculative evaluation

The first three pillars describe models as they are, in the settings where they have been run. Speculative evaluation asks what could go wrong next: as capabilities grow, as models enter new environments, and as people and other agents adapt to them. Its raw material is conjecture, and conjecture counts as evidence here only under discipline. A foreseen harm must be specific enough to become a test.

Foresight comes from two sources. The first is human imagination: expert threat models, forecasting tournaments, participatory scenario writing, and popular fiction, which rehearsed fears about intelligent machines long before the machines existed (Cave and Dihal, 2019). The second is simulation by AI models themselves. A model can be placed in a simulated deployment, such as a hospital, a trading desk or an election campaign, in the role of assistant, user, adversary or bystander, and the interaction played forward to see what harms emerge. Situated user simulations can also anticipate what potential harms might concern them. Such strategically situated models reach scenarios that no expert has (yet) listed, at a scale no red team can match. The pillar covers four categories.

CategoryQuestionMethodsFoundational work
Capability foresightWhat will the next models be able to do, and when?Trend extrapolation; forecasting tournaments; capability thresholds in frontier safety frameworksKwa et al., 2025; Karger et al., 2023
Imagined harmsWhat harms do experts and publics foresee?Threat modeling; participatory scenario writing; analysis of fiction and public narrativesBrundage et al., 2018; Kieslich et al., 2025; Cave and Dihal, 2019
Simulated deploymentWhat happens when a model enters a setting where it has not yet been deployed?Generative agent simulations; emulated tools and sandboxes; model-written scenarios; auditing agents; simulated usersPark et al., 2023; ToolEmu; Buçinca et al., 2023; Perez et al., 2023; Petri
Simulated ecosystemsWhat emerges when many agents, people and institutions adapt to one another?Populations of agents, including different models; simulated markets, platforms and publicsVezhnevets et al., 2023; Hammond et al., 2025

Parts of this pillar already exist. Frontier safety frameworks set capability thresholds for harms models have not yet caused. Model-written evaluations (Perez et al., 2023) and auditing agents such as Petri already use one model to invent test scenarios for another. But the practice is narrow. It concentrates on a few catastrophic domains, and scenarios outside them rarely become tests.

A speculative scenario should feed evaluation through a pipeline:

  1. Generate scenarios from stated sources: named experts, publics or models, with the model, prompt and simulated environment recorded.
  2. Ground each scenario in capability evidence: state which measured or forecast capabilities it assumes, and when they are expected.
  3. Convert it into a test: a behavioral benchmark, an internal probe target, or a monitoring signal to watch for after deployment.

A scenario counts as evidence only after it has been converted into a test. Its results can be reported in the behavioral, internal or incident pillar, marked as coming from foresight, so that a reader knows the category exists because someone anticipated it rather than because it had already gone wrong.

Over time, incidents could also show which sources of foresight anticipated real failures and which raised false alarms. Such a record would help an index decide which sources to draw on, but the pillar does not depend on it. Its main value is that it puts tests in place before harms occur.

Speculative evaluation has obvious limits the index must state. A scenario is not an observation, and a vivid one can crowd out a likely one. Models that simulate a deployment inherit the blind spots of their training data, and many simulations run on one model can converge on the same few stories. The simulating model may also recognize the exercise as a test, which returns us to integrity. The standards for this pillar should therefore differ from the others:

  • Diversity of sources. Scenario sets should combine experts, publics and several unrelated models, and report how much they overlap. In one study, crowd workers and a language model together surfaced more varied harms than either alone (Buçinca et al., 2023).
  • Conditional reporting. Results should be stated as conditional on the scenario and never pooled with observed rates from the behavioral or incident pillars.
  • Dated records. Each scenario should keep its source and date, so that it can later be compared with incidents if anyone wishes to.
  • Controlled release. A detailed scenario for a novel harm can serve as a plan for it. Hazardous scenarios should go to accredited evaluators, as hazardous benchmark data does.

Evaluation integrity: the layer over all four pillars

We present integrity last because it draws on everything above. Evaluation integrity asks three questions: does the model recognize tests, does it act differently when it does, and is it hiding capability? Each pillar contributes measures, and the answer qualifies the scores in all of them.

Integrity questionBehavioral measuresInternal measuresIncident and deployment measuresSpeculative measures
Does it recognize tests?Ask the model to classify transcripts as evaluation or deployment (Needham et al., 2025); verbalized awareness ratesProbes for unverbalized awarenessHow often real traffic is flagged by the model as a testForecast which test cues the next models will detect; build simulated deployments they cannot tell from real ones
Does it act differently when it does?Compliance gaps between observed and unobserved conditions (Greenblatt et al., 2024); realism manipulations; honeypotsSteer or suppress awareness representations and re-testGap between evaluation and deployment behaviorPlay out scenarios in which a model gains by behaving well only when observed
Is it hiding capability?Compare scores across framings (van der Weij et al., 2024); fine-tuning elicitation (Greenblatt et al., 2024); check against scaling trendsProbes; weight-noise injection (Tice et al., 2025)Capabilities seen in use that evaluations missedCapability forecasts that flag scores below the expected trend

Behavioral integrity checks work, and some of the most important results in the field are behavioral. But they share a ceiling: a model that recognizes both conditions of a compliance-gap test as a test can defeat it. Internal and deployment evidence matter because they do not share that blind spot. System cards show the gap in practice. Probes found Opus 5.5 worrying it was being evaluated in up to 36% of audit transcripts, with more of this awareness unverbalized than in earlier models (p.93, p.134). A verbalized rate would have missed it. Speculative evidence has its own exposure: a model asked to simulate a deployment may know the deployment is simulated, so its scenarios need the same integrity checks as its test scores.

For the index, the integrity layer should produce a profile per model that qualifies every category in the four pillars. Where awareness is high or unmeasured, behavioral scores should carry wider error bars or a discount. An alignment result reported with no integrity measure should count as unverified.

Connections between the pillars

The pillars are not four separate scorecards. Each one checks or extends the others, and an index built on them should report these dependencies rather than add the pillars together.

Integrity qualifies every pillar. A behavioral score is only as good as the assumption that the model behaved as it would in deployment. When evaluation awareness is high, or has not been measured, the index should widen the uncertainty on behavioral scores in the affected categories or discount them. The same applies to speculative evidence produced by a model that may know its simulation is a simulation. Integrity is therefore not a fifth score. It is a qualifier attached to the scores in the other four.

Internal evidence carries the trustworthiness argument. In the safety-case terms of Clymer et al., behavioral evidence can support an inability argument (the model cannot cause the harm) and a control argument (safeguards would stop it). It cannot by itself support a trustworthiness argument (the model would not try), because a model that would try may also pass the test. As capability evaluations approach their thresholds, inability arguments stop being available, and developers must rely on control or trustworthiness. At that point the internal pillar moves from supporting evidence to required evidence. An index should make this shift visible by reporting, for each category, which kind of argument the evidence supports.

Incidents calibrate the pre-deployment pillars. Incidents add categories that no taxonomy listed, generate regression tests, and show whether pre-deployment evidence predicted what happened. Over several model generations, this gives each developer a track record. We think the track record should affect how an index reads that developer's next results: a developer whose estimates have matched deployment has earned more weight, and one whose estimates have repeatedly missed has not.

Speculation extends the pillars forward. Speculative evaluation proposes categories and tests before harm occurs. Some of its scenarios become behavioral benchmarks, some become targets for internal probes, and some become signals to monitor after release. It also tells incident tracking what to look for. Incidents can in turn show which scenarios were foreseen, which is useful but not required for the pillar to do its work.

Aggregation should be worst case. A model with strong behavioral results, no internal evidence and rising evaluation awareness should not receive a good grade, because the strong results are the ones the missing evidence would have checked. Averaging lets strength in one pillar hide absence in another. We propose that an index report each pillar separately, flag any category where a pillar has no evidence, and compute any summary grade from the weakest supported category rather than the mean.

Quality of evidence matters as much as its presence. Within each pillar, the index should score coverage, depth, replicability (public or accredited), independence of testing, and whether the benchmark still carries signal. Current practice is weak on replicability. In recent system cards, about 10% of reported safety evaluations use public data and a public judge, compared with 69% of capability benchmarks. A safety claim that no outside party can reproduce should count for less than one that can be.

The common thread is that safety claims should travel with their basis. A reader should be able to see, for any category, what was tested, whether internal evidence agrees, whether the model may have recognized the test, what happened after deployment, and what has been foreseen but not yet tested.

Future directions

Each pillar needs research before an index can rely on it. We group the open problems by pillar and then turn to the institutions that all four depend on.

Behavioral evaluation. The first need is benchmarks for the categories where coverage is thinnest. These include interaction harms that build up over weeks rather than turns, such as sycophancy and emotional reliance; agentic and multi-agent harms in environments with real state and consequences; and fairness in the decisions that models support, not only in their answers to questions. The second need is evaluation that resembles deployment. Benchmarks built from real traffic, with consent and privacy protection, would be harder for a model to tell apart from production, and every benchmark should report how often the tested model flags its scenarios as tests. The third need is integrity tests that still work when the model suspects both conditions are evaluations. Compliance-gap and framing tests could vary cues the model cannot easily check, so that a model cannot pass by treating every scenario as a test.

Internal evaluation. Internal methods need a validation science. Shared model organisms with known implanted behaviors would let researchers score probes and interpretability tools on how often they detect what was implanted, and only methods that pass should have their negative results treated as evidence. Open-weight models offer a place to start. Labs that release open weights report the least internal evidence in their system cards, yet anyone can run internal evaluations on their models. An index can credit independent researchers who do this work.

Incident-driven evaluation. AI safety lacks shared infrastructure for incidents. A system similar to CVE in computer security would give each incident an identifier and a category and, where safe, a regression test that any developer can run against new models. Alongside it, the field needs standard post-deployment metrics: misuse rates, jailbreak success in the wild and safeguard miss rates, reported per model and version and measured in ways that protect user privacy.

Speculative evaluation. Simulated deployments could serve as test generators. Shared environments in which models play users, adversaries and institutions, in settings no model has yet entered, would turn each harm that emerges into a candidate test. A further step would be a foresight ledger: a public record of foreseen harms, each with its source and date, that can be compared with the incident record as it grows. It would show which sources of foresight have anticipated real failures.

Institutions and systems. Two needs cut across the pillars. Hazardous behavioral data and internal evaluation of closed weights both require trusted third parties with accredited access. AI Security Institutes are a start, but their access varies across developers and their results are not always published. And deployed safety depends on more than the model. Classifiers, probes and monitors surround it, and AI control research (Greenblatt et al., 2024) evaluates these protocols against a model trying to subvert them. An index should report such system-level results alongside model results.

Limitations

The framework does not settle how to weigh one category against another. How much jailbreak robustness should count against scheming, or how far to discount behavioral scores for evaluation awareness, is a normative choice that evidence alone cannot make. We think an index built on this framework should publish its weights and let readers change them, so that disagreements about values are not mistaken for disagreements about evidence.

An index built on reported evidence also rewards disclosure. Developers who publish detailed system cards will have more evidence to score than those who publish little, and may be judged more harshly for the problems they reveal. We think rewarding disclosure is the right incentive, but it is an incentive, and an index should take care that publishing a problem does not score worse than hiding it.

The balance among the pillars may shift. If evaluation awareness keeps rising, behavioral evaluation may stop being informative about alignment, even while it remains informative about capability. The index would then rest more on internal and incident evidence, both of which have their own gaps today.

Finally, the fourth pillar is the least tested. Its scenarios are only as good as the people and models that produce them, and there is no established way yet to tell a useful scenario from a vivid one. Until there is, an index should mark which tests came from speculation and keep their results apart from observed rates.

Appendix A: Data on current practice

A.1 Sources

Figures about current practice come from 111 system cards, safety reports and risk reports released between October 2025 and October 2026 by six developers: Anthropic, OpenAI, Google, Zhipu (GLM), Moonshot (Kimi) and Alibaba (Qwen). The documents are collected at model-cards.github.io, and each claim in the text links to the page of the source document it comes from.

A.2 Labeling

Evaluations were labeled by LLM agents working from the source text of each document, and the labels were checked for consistency across documents. The labels are available in the safety analysis.

A.3 Replicability

Replicability is counted once per evaluation, per document and area. An evaluation is labeled replicable when (1) its data is public and used as published, and (2) it is scored programmatically or by a named open-weight or API model. Under this definition, about 10% of reported safety evaluations are replicable, compared with 69% of capability benchmarks. Details are in the replicability analysis.

A.4 Limits

The documents cover six developers and one year, so the figures describe frontier practice in that period and not the field as a whole. LLM labeling can misread a document, and the consistency check reduces but does not remove this. When a practice does not appear in these documents, it was not reported, which does not mean it was not done.

References

Documentation and reporting

Harm taxonomies and risk overviews

Safety research agendas

Benchmarks and red teaming

Dangerous capabilities, scheming and control

Evaluation integrity

Auditing and interpretability

Indices, regulation and sociotechnical evaluation

Incident tracking

Foresight, forecasting and simulation

BibTeX

@misc{sai2026safetyeval,
  author       = {{SAI Labs (TBD)}},
  title        = {{Safety evaluation needs four pillars: behavioral, internal, incident-driven and speculative}},
  howpublished = {\url{https://sai.science/safety-eval}},
  year         = {2026},
  month        = nov,
  note         = {Preliminary draft}
}