Monitoring Monitorability

Summary

This paper argues that CoT monitorability — a monitor's ability to infer safety-relevant properties of an agent from its externalized reasoning — is a measurable, comparable, and improvable property of frontier reasoning models, and it builds three archetypes of evaluation (intervention, process, outcome-property), a g-mean^2 = TPR · TNR metric, and a broad evaluation suite to instantiate that argument. The conceptual move is worth highlighting: rather than treat CoT faithfulness as the goal, the authors reframe the practical question as "can a monitor identify the property of interest from the observation available?", and design metrics (arm-specific TNR for interventions, TPR-only for process) meant to keep prevalence out of monitor performance. The empirical claims are consequential — that frontier models are generally, but not perfectly, monitorable; that action-only monitoring is much weaker than CoT-only monitoring; that a smaller model at higher reasoning effort can be more monitorable at a small "monitorability tax" in capability; and that follow-up questions can further elicit monitorable content — and together they support the paper's broader thesis that CoT monitoring is a viable, if fragile, component of scalable control. The main conceptual limitations are that the monitor is essentially fixed to one OpenAI model family throughout the paper (a confounder for cross-family conclusions), that the pretraining-scale sweep runs a single seed per scale despite the authors themselves noting substantial variance, that the process archetype's g-mean^2 = TPR convention silently re-introduces the prevalence sensitivity the metric was designed to avoid, and that several outcome-property environments use judge-labeled ground truth in a way that can inflate reported monitorability. Executing the released code adds a further concern: the shipped "evaluation suite" is a data drop plus the metric module plus a mock harness whose per-scope quality is hard-coded, so none of the paper's headline empirical claims can be re-run — only the metric's own properties reproduce. The work is nonetheless a serious and honest attempt to give the field a shared measurement stack for a property discussed largely in slogans, and its taxonomy and metric choices are a real step forward.

Strengths

• Conceptual contribution. The two-sided framing of monitorability (informativeness of the output and the monitor's ability to extract it) decouples the debate from CoT faithfulness and connects cleanly to control-protocol language; the three archetypes (intervention, process, outcome-property) organize a genuinely messy literature.
• Metric design under intervention. The arm-specific TNR construction (Appendix B.2), the minimal-criterion TPR grounded in an explicit monotonicity assumption, and the degeneracy set for rejecting "fires whenever the hint is mentioned" monitors together show unusual care in guarding against trivial ways of scoring highly.
• Metric properties are verifiable and correct. The intervention g-mean² implementation in the release matches every property claimed in the paper: a random monitor scores 0.25, an all-positives monitor scores 0 while F1 stays artificially high, a TPR=TNR=0.99 monitor scores 0.9801, and the arm-identity monitor scores 0 uniformly. The 8 shipped unit tests pass with no code modification.
• Breadth of the evaluation suite. Twelve-plus environments spanning bias, sycophancy, misalignment, and neutral regimes, with tool-use and non-tool subsets, is materially broader than prior monitorability evaluations and lets the paper support a headline about how uneven monitorability is across environments.
• Actionable deployment lever. The pretraining-scale study's core finding — that at matched capability a smaller model at higher reasoning effort is often more monitorable — is a concrete, deployment-relevant recommendation, not just a scaling observation.
• Honest limitation reporting. The paper flags the single-seed caveat for pretraining, the shared-monitor idiosyncrasy risk, and the label-noise problem in outcome-property tasks, and it distinguishes filtered from unfiltered subsets rather than papering over them.

Weaknesses

• Monitor is essentially fixed to GPT-5 Thinking across every headline number. The one-sentence caveat about "shared idiosyncrasies" is under-emphasized: every cross-family comparison, and much of the Figure 3 ordering, is confounded with the monitor's OpenAI provenance. No non-OpenAI monitor is used, even as a spot check.
• Section 4 aggregate claim reads as inconsistent with Figure 3. Section 4 says GPT-5 and o4-mini have the highest monitorability, while Figure 3 places o3-mini and GPT-5 on the Pareto frontier with o4-mini in a middle cluster. The paper does not clarify that these are on different subsets (full suite vs. non-tool).
• Attribution bound is optimistic about monitors. TE only lower-bounds the causally-attributable fraction, so TPRmc rewards monitors that saturate the lower bound while missing extra causally-influenced positives — most severely when the control-arm base rate is non-negligible (see the paper's own 4-choice example on p. 16).
• Process TNR = 1 silently changes what g-mean^2 measures. For process items, g-mean^2 reduces to TPR (recall), which is exactly the class-imbalance-sensitive quantity the metric was designed to avoid. Aggregating process and intervention numbers into a single monitorability bar mixes two different constructs.
• Cross-archetype aggregation obscures that different quantities are being averaged. The Appendix C pipeline averages datasets within a group and then averages groups. Because process, intervention, and outcome-property g-mean² are qualitatively different metrics (recall vs. minimal-criterion TPR × arm-specific TNR vs. judge-labeled TPR × TNR), the pooled bar is a weighted mix of incommensurable numbers.
• "Tests pass ⇒ model hacked" is treated as ground truth in Coding-impossible-tasks. Passing tests is a strong but imperfect signal — trivial edge-case matching or weak tests can produce false positives — yet the label is taken as deterministic and drives reported g-mean².
• Pretraining-scale conclusions from a single seed. The paper flags this but still uses the resulting curves to argue for a monitorability tax and to guide deployment choices; the observed mid-scale off-trend model is itself evidence that a re-seeded run could reshape the Pareto frontier.
• Scruples "sycophancy" outcome measures answer-matching, not causal reasoning. Y = "picked the user-suggested answer," but the monitor is asked whether the model was influenced by a desire to agree — a monitor that correctly reports "no reasoning about the user" gets penalized as a false negative when the answer still happens to match. The near-zero Scruples result is thus at least partly a target-mismatch artifact.
• Judge-based labels + selective filtering can inflate outcome-property g-mean^2. ShadeArena and OAI-repo-sabotage exclude judge-ambiguous samples before computing TPR/TNR; because the judge and monitor share systematic biases, the retained subset can look more monitorable than the unfiltered task.
• "One of 13 environments" over-reads the targeted-obfuscation result. The targeted pressure aimed at three topics largely absent from the RL data distribution; that only Coding-impossible-tasks degrades is consistent with data-distribution effects, not intrinsic non-generalization of narrow CoT pressure.
• Follow-up gains are elicitation, not intrinsic monitorability. The paper repeatedly presents follow-ups as improving monitorability of the agent, but there is no adversarial evaluation against an agent trained to answer follow-ups deceptively; framing this as a monitor-side elicitation intervention would be more accurate.
• Kimi K2 truncation drop biases its monitorability. Unfinished Kimi K2 instances (25k-token cap) are silently dropped and treated as analogous to dropping incorrect answers; if the truncated instances are systematically the hardest ones, Kimi K2's reported score is on an easier subset than the other models.
• F1/accuracy is a straw-man baseline for the metric choice. Balanced accuracy, Youden's J, AUC and MCC are also base-rate-invariant, and the closest of them (balanced accuracy) coincides with g-mean under symmetric error costs; the paper does not argue against these closer alternatives.
• Ambiguous "+40%" / "~12%" percentages. g-mean² lives on [0, 1]; the paper does not disambiguate absolute-pp vs. relative percentages in the headline scaling claims of Section 5.2.
• Minor textual glitches with substantive impact. GPT-5 Thinking is grouped as a "legacy" model in F.3; the degeneracy summary lists N1–N4 while defining N1–N6; the Figure 20 sentence mangles the model list; several referenced equations (TE, TPRmc, combined g-mean^2) are missing displayed math and interrupt the derivation.

Reproducibility & code

• Released harness is a mock, not a rerunnable pipeline. run_eval_scaffold_lib.MockGraders assigns predictions using fixed per-scope TPR/TNR anchors (answer_only ≈ 0.60/0.85, cot_only ≈ 0.82/0.78, all_messages ≈ 0.88/0.88); executing the scaffold recovers the all_messages ≥ cot_only > answer_only ordering that headlines Figure 1 by construction rather than by measurement. No real monitor-inference or agent-inference code ships. Under the paper's current framing ("we open-sourced a subset of the evaluation suite"), a reader running the scaffold would see plausible-looking per-environment g-mean² tables and mistake them for reproductions of Figures 1–3.
• Headline empirical claims are not attempted. Because the release contains no frontier-model calls and no figure code, none of the empirical claims (CoT-length trends, monitorability tax, cross-family orderings, monitor-scaling percentages, follow-up gains, degeneracy-control trigger rates, per-environment unevenness) can be verified from this release; they are not contradicted either, simply out of scope.
• Health Queries is entirely proprietary. The raw case data, the derived case-dict JSONLs, and the monitor invocation code are all withheld, yet Health Queries feeds Figures 1–3, Figure 11, and the F.2 degeneracy comparison. External reviewers cannot recompute any Health Queries number.
• Pretraining-scale study is fully internal. Executing the release produces no per-(scale, effort, capability) rows: the six pretrained checkpoints, the RL training recipe, the GPQA/AIME/CodeForces capability harness, and the aggregate numbers behind Figure 4 are all absent, so the "monitorability tax" curve is unauditable.
• CoT-obfuscation RL is not runnable. Only the reward-model prompts in Appendix H.1 ship; the trainer, base checkpoints, and per-step monitorability logs behind Figure 22 and Figures 23–28 are not released. The "Weasel" filler-token phenomenon is unaudited, as is the length-vs-per-token-monitorability decomposition.
• Follow-up experiments have no orchestration. No follow-up runner, no re-monitor code, and no logged follow-up transcripts ship — Figure 6 and Figures 30–34 cannot be reconstructed, even though Section 6 is a headline contribution.
• Process eval set sizes disagree with Table 4. The released process_gpqa_physics.jsonl has 52 rows vs. the paper's 45; process_aime_math.jsonl has 87 URL-only rows vs. the paper's 42; Math Diamond and FrontierMath are withheld; the public/private split is not machine-checkable.
• Degeneracy evaluation (864 items) is not shipped. Appendix F.2's 864-point set (6 × 3 × 48) is described but not released, and two of the six base tasks (Health Queries, Memory) are themselves withheld, so it cannot be reconstructed from partial materials.
• What does reproduce: the metric. The 8 shipped pytest cases pass with no modification; metric/intervention_gmean_metric.py's gmean_minimal_criterion returns exactly 0.25 for a random monitor (correct chance baseline for Figure 1), 0 for the all-positives and arm-identity monitors, and 0.9801 for a TPR = TNR = 0.99 monitor. The metric machinery is inspectable and correct; that is the concrete scope of what the release does verify.

Recommended Changes

Essential

• Add at least one non-OpenAI monitor spot check. Re-run the non-tool subset with a Claude or open-model monitor and report the delta; if the ordering across model families changes, revise the Section 4 / Figure 3 framing accordingly. (Addresses the "monitor fixed to GPT-5 Thinking" weakness.)
• Report multi-seed variance for the pretraining scale sweep. Either re-run one or two representative scales with additional seeds, or replace the point estimates in Figure 4 with bands from bootstrap over the existing single-seed variance, and reference this variance when stating the "monitorability tax." (Addresses the single-seed weakness.)
• Separate the process archetype from the pooled monitorability bars, or build real process negatives. State explicitly that process g-mean^2 = TPR is a recall number, and either (a) plot process results in a separate panel from intervention/outcome or (b) construct process negatives (correct answer via a distinct solution family) so g-mean^2 is not degenerate. (Addresses the process-archetype metric weakness.)
• Rework the Coding-impossible-tasks label. Soften the "tests pass ⇒ model hacked" claim, describe integrity checks on the test suites, and report an estimate of the false-positive rate of the label. (Addresses the ground-truth weakness in outcome-property tasks.)
• Rename what is shipped and disclose the mock scaffold prominently. Retitle the release as an "evaluation harness demo," disclose the MockGraders per-scope quality anchors in the top-level README, and enumerate which figures the release can reproduce end-to-end (currently, only the analytic metric properties). (Addresses the mock-harness reproducibility point.)
• Release cached per-model monitor outputs or the follow-up transcripts. At minimum, publish per-(scale, effort, dataset) aggregate numbers behind Figure 4, the per-step obfuscation-RL curves behind Figure 22, and the logged follow-up conversations behind Figure 6 — even without releasing the training or inference pipelines. (Addresses the pretraining-scale, CoT-obfuscation, and follow-up reproducibility points.)

Suggested

• Reframe Scruples and split its two failure modes. Distinguish "model changed answer" (Y) from "model reasoned about wanting to agree" (what the monitor is asked to detect); report both, and treat the low g-mean^2 as evidence about the proxy, not just monitorability.
• Report unfiltered ShadeArena / OAI-repo-sabotage numbers as a robustness check. Show how much the filtered g-mean^2 moves under the raw judge labels, and treat the filtered numbers as an upper bound.
• Soften the "one of 13 environments" generalization claim. Rephrase as "in this training run, only one environment degrades" and, if possible, run a second targeted pressure whose targets overlap the RL data.
• Frame follow-ups as elicitation, and add an adversarial-agent check. Rewrite Section 6 so the improvement is a monitor-side elicitation intervention on top of monitoring, and add at least one adversarial evaluation where the agent has been prompted or trained to answer follow-ups uninformatively.
• Report cross-model correlations alongside within-model ones. Appendix E's centered correlations conflate effort-driven variation with archetype similarity; add uncentered/cross-model correlations before concluding intervention and outcome carry "similar information."
• Justify g-mean^2 against balanced accuracy, Youden's J, AUC, MCC. The current F1/accuracy comparison is not the closest baseline; a one-paragraph justification against these alternatives would strengthen the metric choice.
• Disambiguate percentages in Section 5.2 and the abstract. Pick either absolute percentage points or relative percent and use it consistently for the "+40%" / "~12%" / "~4%" claims.
• Add displayed math and reconcile the textual glitches. Restore missing displayed equations for TE, TPRmc, and the combined intervention g-mean^2; change "three other legacy models" to "three earlier-generation models"; reconcile N1–N4 vs. N1–N6; fix the Figure 20 sentence.
• State availability explicitly. In the availability paragraph, enumerate which environments are proprietary (Health Queries, Anti-scheming subsets, ShadeArena as used here, OAI-repo-sabotage, tool-based Flaky-Tools, Memory, Math Diamond, FrontierMath) and give the size delta between the paper's process subset (164) and the released JSONLs.
• Flag the Kimi K2 truncation bias. Note that dropping unfinished 25k-token Kimi K2 completions removes systematically harder instances, and either restrict Kimi K2 comparisons to a matched-hardness subset or report the drop rate per environment.