Information Flow Reveals When to Trust Language Models

Summary

The paper proposes a white-box uncertainty-quantification (UQ) method for retrieval-augmented generation that treats prediction reliability as a function of how a language model's internal information flow attends to query-relevant context tokens. Building on the information-flow decomposition of Ferrando & Voita, the authors introduce two constructs: an Emergence Order (extracted by an Auto-Emergence algorithm that walks the layer-wise contribution matrices $\{\mathbf{C}^{(l)}\}$ backward from the last input token) and a Contribution Layout (aggregating all admissible input-to-output paths under the causal-mask monotonicity constraint). These layouts are compared, via rank-biased overlap at token/word/phrase granularities, against a Shapley decomposition of an external reranker's relevance score (Simulatability), and their concentration against a uniform distribution via KL (Concentration). Twelve resulting features are fed to an XGBoost calibrator. The conceptual move — treating grounding as a sequential propagation quantity rather than a single-layer snapshot — is well motivated by the paper's critique of prior white-box UQ methods that collapse layer structure. Two conceptual concerns dominate: circularity (the phrase-level partition used at one simulatability granularity is itself derived from the reference relevance layout, so agreement is partly baked in), and interpretive overreach (the causal reading of a small $n=100$ top-vs-bottom token ablation as "functionally necessary" outruns the design, and Section 6.4's framing of the fitted 12-dimensional XGBoost model as "not a performance-enhancing calibrator but a necessary scalarization function" overstates what a learned mapping does relative to already-scalar baselines). The executed replication complicates the empirical picture: on the compute-forced tiny test splits that ran, Ours's mean AUROC (~0.699) is close to the paper's 0.709, but the abstract's comparison flips — Focus recomputes at ~0.744 — and the faithfulness figure shows no sharp top-vs-bottom contrast (both ablations retain ~30% correctness on the one cell rerun). The calibration story (best ECE among calibrated methods) and the info-flow-features-vs-ranker-signals ordering hold directionally on LLaMA cells, so the mechanism claims survive at small scale; the reported rankings against baselines do not.

Strengths

• Conceptual contribution. Treating grounding as sequential layer-wise propagation, and formalising it via the Auto-Emergence backward trace plus the all-admissible-paths Contribution Layout, is a principled refinement of prior single-layer or max-pool white-box UQ, and the two views (direct evidence vs. distributed context-building) are cleanly complementary.
• Multi-view feature construction is well-motivated. The pairing of simulatability (alignment with an external relevance signal) with concentration (peakedness of the layout) captures orthogonal aspects of contextual grounding, and the ranker-swap and RBO-$p$ sensitivity studies argue against dependence on any single ancillary choice.
• Faithfulness-oriented framing. The top-vs-bottom token ablation of Section 6.2 directly targets whether identified tokens actually drive predictions, a more useful sanity check than correlational analyses alone (even if the interpretation is overstated and the replicated figure does not show the claimed contrast — see Weaknesses).
• Calibration is reported alongside discrimination. Appendix I reports ECE and accuracy-at-0.5 for calibrated methods, and this is the part of the paper that reproduces most robustly in the veritas run — Ours has the lowest ECE on every completed cell.
• Post-hoc calibration is not the source of gains (qualitatively). Tables 11–13's finding that scalar baselines do not benefit from XGBoost calibration reproduces qualitatively on the one cell rerun (only Regular Entropy marginally improves; PPL/P(True)/Attention collapse to 0.5; Focus and Semantic Entropy degrade), which supports the claim that the improvement comes from the underlying signals rather than the calibrator.

Weaknesses

• Headline AUROC ordering flips under executed replication. The abstract's core comparison (0.709 vs 0.676) does not survive re-execution: the strongest baseline recomputed at ~0.744 on the four overlapping cells, and Ours wins only 2 of 5 completed cells. The margin (0.033 AUROC) is smaller than the ranker-swap spread in Table 10 (~0.04–0.05) and smaller than the noise induced by the compute-forced tiny test splits, so the paper needs per-cell confidence intervals before this ordering can be relied on.
• Faithfulness figure does not show the claimed contrast. On the reconstructed Figure-6 pipeline, removing bottom-ranked tokens degrades correctness roughly as much as removing top-ranked tokens (~30–31% vs ~32–33% retained). Combined with the missing random-token control, the missing per-sample confidence intervals, and the restriction to already-correct samples, the causal-necessity reading of Section 6.2 is not supported by the data.
• Circularity in phrase-level simulatability. Phrase units are built by clustering words whose Shapley values in $\mathbf{R}^{\text{layout}}$ are correlated — but $\mathbf{R}^{\text{layout}}$ is precisely the reference against which phrase-level RBO is then computed. The partition therefore encodes information about the target ranking, likely inflating phrase-level scores. Section 6.3 acknowledges a related ranker-bias worry but does not address this particular feedback loop.
• Overreach in the "functionally necessary" claim. With $n=100$ correctly predicted samples per dataset, ablating the top-5 tokens flips a "majority" of predictions, which means up to nearly half survive. The paper reads this as "direct causal evidence" of "functional necessity," but the experiment lacks (a) a random 5-token ablation control, (b) confidence intervals, and (c) any test on incorrectly predicted samples, where a UQ signal is arguably most needed.
• OOD generalization has an alternative explanation. The Table 2 observation that calibrators trained on different sources reach similar test-set performance is equally consistent with the calibrator being nearly a no-op (i.e., the raw features carry the signal on their own). A simple untrained aggregation baseline would distinguish "features generalize" from "calibrator adds little."
• Interpretive framing of Section 6.4. Calling AUROC "inherently ill-suited" to multi-dimensional signals overstates a routine requirement, and framing the XGBoost + Optuna fit on 12 features as "not a calibrator but a necessary scalarization function" masks a real asymmetry: scalar baselines cannot exploit feature interactions the way Ours can. The executed replication further shows the baseline-side calibrator is fit with asymmetric hyperparameters, so the "same calibration procedure" defence does not hold end-to-end.
• Reference-set semantics of the correctness pipeline. The pipeline builds one reference statement per sample and scores against it with HHEM at threshold 0.5. The justification invokes the multi-valid-answer case ("1939" vs "late 1930s"), but nothing in the described procedure enumerates or unions over multiple valid answers — the defence given is really about semantic-vs-token matching, not multi-answer coverage.
• Generation-length constraints under-specified. A 5-word prompt cap, a 10-token decoding cap, and a first-period truncation rule interact ambiguously, and the completion criterion for the "discard incomplete samples" rule is not stated — which determines which subset the headline AUROC is averaged over.
• Empirical scope. All three QA models are in the 3B–8B range, and the Gemma-3-4B port produced degenerate all-zero contributions in the veritas run — evidence that even at the current scale the method is model-family-sensitive. Extending to at least one 30B–70B open-weight model would materially strengthen the generality claim.
• Presentation issues in the main tables. Table 1's MS MARCO block collapses AUROC and AUPRC into a single interleaved cell; Table 2 has a similar problem; Appendix F's stability table (Table 4, "Reported values are AUROC means ± standard deviations") in fact reports AUPRC values.
• Non-monotonic AUPRC spike in the p-sweep. Table 3's LLaMA-3.2 SQuAD2.0 AUPRC at $p=0.7$ ($0.855$) is a ~0.03 spike relative to its neighbours ($\sim 0.827$–$0.831$), and this specific value is the one featured in Table 1. Either the sensitivity is real (contradicting the surrounding robustness claim) or the value is a transcription error.
• Ranker-bias attribution and the "without substantial degradation" summary. The AUROC increase after human filtering is read as pure ranker bias, but hard-case discarding is an equally consistent explanation. The Table 10 ranker swap shows ~0.04–0.05 AUROC spread across rankers — larger than the headline improvement over Focus — and the smaller BGE ranker sometimes beats Qwen3-Reranker-8B; "limited sensitivity" glosses this.
• Auto-Emergence algorithmic details. Tie-breaking and the interaction between cross-token flow extraction and the self-contribution sweep for already-ranked $j$ are not specified. Appendix A's supporting empirical claim about self-contributions being selected first is asserted without measurement.

Reproducibility & code

• Headline aggregates. The proposed method's mean AUROC over the five completed model×dataset cells (~0.699) is within ~0.01 of the paper's 0.709, but the recomputed Focus baseline averages 0.744 on the four cells it covers, and Ours is best on only 2 of 5 completed cells. The abstract's "exceeding the runner-up" ordering therefore does not reproduce on this run. The Mean-Rank column of Table 1 cannot be regenerated at all: evaluation/plot.py never computes it.
• Faithfulness figure. On the one cell the reconstructed Figure-6 script produced (LLaMA-3.2-3B / SQuAD2.0), removing bottom-ranked tokens degrades correctness roughly as much as removing top-ranked tokens (retained-correct ~30–31% vs ~32–33%) — the opposite of the claimed sharp contrast. The script itself was author-truncated and had to be reconstructed, the 100-sample selection is unseeded, and no random-token control is implemented.
• Baseline pipeline is a placeholder. baselines/readme.md marks the baseline main script as PLACEHOLDER and none of the eight baseline UQ methods (PPL, P(True), Regular/Semantic Entropy, Attention Score, Focus, KnowingMore, Utility Ranker) has an implementation in the shipped code. Combined with missing baselines_test.json artifacts, the head-to-head comparison in Table 1 cannot be regenerated end to end.
• Critical sampling bug and 22 other required patches. As shipped, the sampling took a contiguous first-N slice of a file sorted by answerability, so at small N the calibrator collapsed to constant 0.5 (AUROC 0.5000). The veritas run had to introduce a stratified sampler across proposed/inference.py, Ours/llama/main.py, Ours/gemma/gemma.py, and baselines/baseline_data_prepare.py; reconstruct evaluation/faithfulness_comparison.py; patch a NaN-producing simplex normalization; remove an extra np.log that turned every PPL score into NaN; fix a ZeroDivisionError in the Focus baseline's IDF; and reconcile a silent 40% sample misalignment on MS MARCO. The Gemma-3-4B port carries three Gemma-specific bugs absent from the working LLaMA path.
• Model paths and environment are placeholders. Every model path is MODEL_PATH_PLACEHOLDER, no requirements.txt/environment.yml is shipped, and hard-coded device_num values differ across scripts (5 in result_preprocess.py vs 8 in Ours/llama/main.sh).
• Ranker-bias analysis is not reproducible. evaluation/human_vs_reranker.py loads manual_filtered.pt (the boolean tensor of human labels that produces the 94%/90%/84% retention numbers), but this file is not in the repository, no annotation task instructions or agreement statistics are shipped, and the script hardcodes obtained_num=500 while only 200 samples were produced in the veritas run.
• Stability-analysis multi-run harness is absent. Appendix F reports variances across three seeds, but no driver in the repo runs the pipeline more than once and neither the SHAP explainer nor Optuna is seeded, so the reported ~$10^{-4}$–$10^{-8}$ variances cannot be independently regenerated.
• Baseline-side calibrator is not applied under symmetric conditions. evaluation/baselines_calibratior.py uses $n\_\text{trials}=10$ for Optuna, whereas proposed/calibrator.py uses $n\_\text{trials}=50$; a dtrain_full (train+val) matrix is constructed but then unused, since xgb.train is called with dtrain=dtrain (train only). The Section 6.4 claim that the "same calibration procedure" was applied to baselines therefore does not hold in the released code, and equalising it could change Tables 11–13.
• Ranker-swap experiment was not exercised. Only Qwen3 was ever invoked in the veritas run; SHAP_MiniLM_L12.py and SHAP_BGE-v2-m3.py exist but were never called, in part because proposed/calibrator.py writes the --reranker argparse choices as 'Qwen3' 'MiniLM' back-to-back, so Python silently concatenates them and BGE is not selectable from the CLI. As a result all of Table 10 is unreproduced.
• Figure 7 permutation importances are computed but never emitted. proposed/calibrator.py computes imp_auroc/imp_auprc per feature at lines 186–187 but the script ends immediately after — the per-feature drops are never saved, printed, or plotted, so Figure 7 is not regenerable.
• What does hold up under execution. The proposed method's AUROC magnitude on the completed cells is within a few percent of the paper's values on 3B SQuAD2.0 and 8B SQuAD2.0; Ours has the lowest ECE on every completed calibrated cell (matching the paper's calibration story qualitatively, though ECE magnitudes are 2–5× higher than reported); the info-flow-features-only vs ranker-signal-only ordering (Table 6) reproduces directionally for the LLaMA cells; and the qualitative claim that post-hoc calibration does not rescue scalar baselines (Tables 11–13) is confirmed on the one cell that ran.

Recommended Changes

Essential

• Report per-cell confidence intervals for Table 1 and revisit the headline claim. Given that Focus recomputes to ~0.744 in the executed replication (vs 0.709 for Ours) and the 0.033 gap is smaller than the ranker-swap spread, add bootstrap CIs on every cell and re-state the abstract accordingly. This is the direct fix for the headline-AUROC-ordering flip in Reproducibility & code.
• Rerun the faithfulness ablation with controls and revise Section 6.2. Add (a) a uniformly-random 5-token ablation baseline, (b) an ablation on incorrectly-predicted samples, (c) a seeded 100-sample selection with per-cell confidence intervals, and (d) run all 3×3 cells. Soften "functionally necessary" to "strongly influential." This addresses both the interpretive overreach and the reproducibility gap where the claimed sharp contrast did not appear.
• Break the phrase-level circularity or measure the inflation. Either construct the phrase partition from a signal independent of $\mathbf{R}^{\text{layout}}$ (syntactic chunking or a separate embedding-based clustering) and re-run the phrase-level simulatability, or add a random-phrase-grouping control that bounds the artefactual gain.
• Re-run Section 6.4 with symmetric calibrator settings. Equalize $n\_\text{trials}$ between baselines_calibratior.py and proposed/calibrator.py, fix the ignored dtrain_full bug, and re-report Tables 11–13. Retitle Section 6.4 to acknowledge the residual 12-dim-vs-1-dim asymmetry.
• Release the missing repository pieces needed to regenerate the headline table. Ship the baseline UQ implementations (or the precomputed baselines_test.json files), a requirements.txt/environment.yml, real model paths (or a clearly documented placeholder-substitution recipe), seeded SHAP/Optuna calls, manual_filtered.pt plus the annotation instructions, a multi-seed driver for Appendix F, a seed for the faithfulness sample selection, the Figure-7 permutation-importance persistence and plotting code, and fix the --reranker argparse concatenation bug so the BGE row of Table 10 is reachable.
• Fix the critical stratified-sampling bug and the Gemma port before publication. The as-shipped contiguous first-N sampler and the three Gemma-specific bugs cause the calibrator to collapse to AUROC 0.5 and the Gemma information-flow port to emit all-zero contributions. Ship the stratified sampler as the default and repair the Gemma path, so a reproducer does not need to write ~20 patches to obtain non-degenerate metrics.

Suggested

• Add a third row to Table 2 for an untrained aggregation baseline. A simple mean/sum of the 12 features would distinguish "features generalize" from "calibrator adds little."
• Fix the table rendering. Regenerate Table 1 and Table 2 so AUROC and AUPRC live in visibly separate columns for every dataset; correct Table 4's caption ("AUROC means") to match its contents (they appear to be AUPRC values) or replace the numbers with true AUROC values.
• Clarify the correctness pipeline. Either replace the "some questions admit multiple valid answers" motivation with the correct semantic-vs-token framing, or genuinely evaluate against the max over reference statements for datasets with multiple gold answers.
• Pin down the completion criterion. State whether a sample is completed on first period, EOS, or when the 10-token cap is reached, and report the discarded-sample fraction per dataset/model.
• Add a strong-model check. Report at least one row of Table 1 on a 13B+ (ideally 30B+) open-weights model so the "applicable to open-source LLMs" positioning has empirical support at scale.
• Sanity-check the Table 3 $p=0.7$ SQuAD2.0 AUPRC spike and either explain the non-monotonicity or correct the value; state how $p=0.7$ was chosen (validation-set selection vs. a-priori).
• Extend the Figure 7 ablation to include word/phrase-level KL scores, or motivate the token-only concentration explicitly.
• Report per-dataset base-QA accuracy alongside AUROC so readers can contextualize the 0.709 average.
• Reconcile the Attention Score baseline description with the Related Work description of Sriramanan et al. (2024), and clarify which variant is implemented.
• Tighten Auto-Emergence. State the tie-break rule and the interaction between cross-token flow extraction and the self-contribution sweep when $j$ already has a rank, and add a small measurement backing Appendix A's claim that self-contributions dominate the pool.
• Soften the ranker-sensitivity summary in Section 6.3. "Limited sensitivity" is not accurate given ~0.04–0.05 AUROC swings; a phrasing like "moderate and non-monotonic sensitivity" reflects Table 10 more honestly.