Privacy-Aware Video Anomaly Detection: Guided Orthogonal Projection and a Comprehensive Evaluation Framework

Summary

The paper proposes the Orthogonal Projection Layer (OPL) and its face-guided variant (G-OPL) as a lightweight, differentiable route to privacy-aware video anomaly detection (VAD). OPL learns a rank-k orthonormal basis $\mathbf{Q}$ via QR and projects features onto the orthogonal complement of $\mathrm{span}(\mathbf{Q})$, stripping nuisance directions. G-OPL adds a cosine-alignment term that drives $\mathbf{Q}$ toward face-related directions using only a weak face-presence signal and the mean backbone embedding of face crops from RetinaFace (or a controlled Georgia Tech Face set), avoiding adversarial training. The authors add three diagnostics — SSC (subspace capture), ARD (KL divergence between raw and projected anomaly-score densities), and PD/FPD (linear-probe accuracy for face presence) — and evaluate on five VAD benchmarks plus CHAD, integrating the modules into RTFM, MGFN, TEVAD, and EGO, with extensive layer-placement and multi-attribute ablations.

The contribution matters. VAD is increasingly deployed where representation-level identity leakage matters, and adversarial gradient reversal is brittle; a QR-based orthogonal-complement projection guided by cheap face-presence signals is a useful design point. Adding G-OPL rarely damages AUC and often improves it, while lowering the paper's privacy metrics and an external ArcFace probe.

The main conceptual limitation is that the gap between mechanism and rhetorical framing is not closed. The supervision is a mean face embedding (which washes out per-identity directions) yet the paper repeatedly claims to remove "biometric identity" and asserts that the projection "guarantees that downstream layers never observe identity-bearing directions" — a claim contradicted by non-zero FPD in Table 4 and 45.3% face-recognition on inversions. Appendix E establishes only partial linear separability, not orthogonality; Proposition E.3 is close to tautological and its Eckart–Young appeal is misapplied. Algorithms 1 and 2 enforce $\mathbf{Q}^\top \mathbf{Q} = \mathbf{I}$ very differently — hard QR in OPL, only a soft Frobenius penalty in G-OPL — so the "orthogonal projection" is not exact during G-OPL training. Combined with several table-level inconsistencies and reproducibility gaps documented below, these weaken the paper's stronger claims.

Strengths

• Conceptual contribution. A learnable, differentiable, QR-stabilised orthogonal-complement projection guided by a cheap face-presence signal is a clean, adversarial-free alternative to gradient-reversal-style privacy, and it plugs cleanly into existing weakly supervised VAD backbones. The choice to operate at the feature level (rather than pixel- or dataset-level) is well motivated and fits the deployment scenario described in Appendix A.
• Efficiency. Table 6 and Table 18 make a credible case that the module is cheap: ~7% latency and <3% memory overhead on RTFM, with a favourable privacy-per-GFLOP curve.
• Breadth of evaluation. Five VAD benchmarks (ShT, UCF, CUHK, Ped2, MSAD) plus CHAD, four base VAD models (RTFM, MGFN, TEVAD, EGO), a placement grid (Tables 20–29), multi-attribute suppression (Tables 7–8), unreliable-supervision robustness (Table 13), and a cross-domain check (train ShT / test CUHK) collectively make the empirical envelope wider than most privacy-preserving VAD papers.
• Diagnostic framework and external cross-check. Introducing SSC / ARD / PD/FPD and validating against ArcFace and a simplified VISPR probe is more thorough than the field's norm. The Fig. 6 $\mathbf{QQ}^\top$ visualisations and Fig. 1/Fig. 5 UMAPs contribute genuine interpretability.
• Practical recipe. Appendix K/L (Table 10) gives a per-dataset hyperparameter recipe and identifies G1O0 as a sensible default, which is unusually actionable for a submission of this scope.

Weaknesses

• Overclaimed guarantees. The claim that the projection "guarantees that downstream layers never observe identity-bearing directions" and the consequent bullets ("identity cannot be recovered", "classifier probes fail to retrieve identities") are contradicted by the paper's own numbers — several G-OPL cells in Table 4 keep FPD at 0.58–1.0, and the Appendix B inversion experiment reports 45.3% recognition on G-OPL inversions. A rank-k linear projection cannot guarantee identity removal even in principle; it removes only what lies in $\mathrm{span}(\mathbf{Q})$.
• Mechanism–claim gap. Guidance is a mean backbone embedding over faces present in a clip (or over 50 Georgia Tech subjects), which by construction suppresses per-identity variance and picks up a shared "faceness" direction. The paper repeatedly frames this as removing "biometric traits" and preventing "re-identification"; that is stronger than what averaging can support. See also the ArcFace probe, which uses $\hat{\mathbf{f}}^{(i)} = \mathbf{Q}\mathbf{Q}^\top \mathbf{f}_{\text{attr}}^{(i)}$ (the captured component, not the released residual $(\mathbf{I}-\mathbf{Q}\mathbf{Q}^\top)\mathbf{f}$).
• Inconsistent orthogonality enforcement. OPL (Algorithm 1) obtains $\mathbf{Q}$ from QR each forward pass, so $\mathbf{Q}^\top \mathbf{Q} = \mathbf{I}_k$ exactly. G-OPL (Algorithm 2) treats $\mathbf{Q}$ as a directly learnable parameter with only the soft penalty $\|\mathbf{Q}^\top \mathbf{Q} - \mathbf{I}_k\|_F^2$, so $\mathbf{Q}\mathbf{Q}^\top$ is not idempotent during training and $\mathbf{I}-\mathbf{Q}\mathbf{Q}^\top$ is not a true orthogonal projector — undermining every "geometric guarantee" claim in the G-OPL setting.
• Metrics conflated. SSC (a subspace-capture diagnostic, higher is better) and ARD (defined by the paper as a utility-preservation metric) are grouped together with FPD under "identity leakage" ("our method reduces identity leakage across both our proposed metrics (SSC/ARD/FPD)…"). This is directionally wrong for SSC and category-wrong for ARD. SSC also grows to 1 as $k \to d$ regardless of alignment, so cross-config SSC comparisons need matched k and a random-Q null.
• Theoretical appendix does not do the load-bearing work. Assumption E.1 gives only partial disentanglement; Proposition E.3 assumes what it claims to establish (existence of a low-error linear classifier) and the appeal to Eckart–Young misapplies a reconstruction-optimality theorem to a classifier-derived subspace. The method's utility argument requires (approximate) orthogonality between anomaly-relevant and sensitive subspaces, which the appendix never bridges to.
• Overstated aggregate performance claims. The Table 3 caption asserts "consistently outperforming or matching top baselines" but the Overall column shows only one of the five proposed variants exceeding the best baseline (EGO 87.3); two clearly underperform. The Appendix M captions for Tables 22, 23, 26 similarly assert G1O0 "consistently achieves top results" while their own numbers place G1O0 middling on many anomaly types.
• Numerical inconsistencies across tables. (i) Tables 27 and 28 report different Overall AUCs for the same RTFM MSAD configurations (e.g., G1O1 84.4 vs 86.6, G1O5 86.2 vs 86.3). (ii) Table 28's CUHK column appears to have AUC and AP transposed from G3O3 onward (AP > AUC by ~20 pp). (iii) Table 20's (B1C1) and (ALL) rows share byte-identical Vandalism/Water/Overall cells despite differing elsewhere. (iv) The VISPR headline 0.63 → 0.27 does not match Table 16's 9-attribute mean of ~0.33 (0.27 matches only the 7 face-conditioned attributes).
• Claim–evidence mismatches in prose. "Performance improves as k increases from 2 to 16, peaking around k = 4" is internally contradictory (should read "from 2 to 4"). "AUC remains stable (85.3 → 84.4)" anchors AUC at the Face row while leakage (73.3 → 49.5) is anchored at Baseline. The "large AUC gains (+8.4%, +3.5%)" cited on ShT/CUHK correspond only to the MGFN backbone (with an unusually low ShT baseline of 75.3); the RTFM gain on ShT is +0.5 pp.
• Faces vs. anomalies — tension. Section L reports MI(face, anomaly) < 0.02 on ShT and CUHK and calls facial information "essentially orthogonal to anomaly semantics", yet the same appendix attributes "large AUC gains" to removing face–anomaly spurious correlation. If MI $\approx$ 0 there is by definition little spurious signal to remove; the mechanism deserves either a mediating explanation (e.g., a specific model artefact that latches on to an uninformative signal) or reconciliation.
• Multi-attribute construction under-justified. The multi-attribute loss reuses one shared low-rank $\mathbf{Q}$ across attributes that need not be near-parallel (face, torso, clothing colour, gait), and "concatenate multiple weak attribute embeddings" (Sec. 3.2, echoed in Sec. 4.2) changes the ambient dimension without discussion. Whether k scales with attribute count is not addressed.
• Sub-chance FPD values. Several G-OPL FPD entries fall to 0.01–0.09 on a binary detection task, well below any plausible base rate for a trained probe that could just predict the majority class. Either the reported quantity is not classification accuracy in the ordinary sense or class balance / label handling is nonstandard; the definition and numbers must be reconciled.
• Practical k guideline contradicts optimal k. The heuristic "$k \approx$ 2–5% of the feature dimension (e.g., $k \in$ [16, 50] for 1024-d features)" is inconsistent with the paper's own peaks at k = 4 (ShT, MSAD) and k_G-OPL = 8 (Ped2) on 1024/2048-d I3D features — actual optima sit at ~0.2–0.8% of the feature dimension.

Reproducibility & code

The submission is accompanied by released code (paper–code mode: veritas READ the release without executing it). Coverage is uneven: the algorithmic core reproduces well from paper + code, but the empirical wrapper leaves many degrees of freedom that neither the paper nor the released material closes.

• Algorithmic core is well documented. Algorithms 1 and 2, Equations 1–6, and Table 10 together provide a full recipe for the projection modules (batch size, epochs, k, $\lambda_{\text{orth}}$, $\lambda_{\text{face}}$, G/O counts per dataset). The QR-based OPL and the cosine + Frobenius G-OPL loss can be implemented directly from the paper.
• Training details missing from the recipe. Table 10 omits optimizer, learning rate, weight decay, LR schedule, and random seed. No per-seed variance is reported anywhere. Given that the headline improvements on ShT (RTFM 96.8 → 97.3) are within typical seed-to-seed noise for these backbones, exact reproducibility of the headline cells is not certain.
• Probe protocols underspecified — this affects the paper's central privacy metric. PD/FPD is defined via "a linear SVM" but leaves probe class, train/test split, class-balance handling, and hyperparameters open. Since FPD drives Tables 4, 5, 6, 9, 13, 14, and 17, and since several G-OPL FPD values fall implausibly below binary chance, the missing protocol is load-bearing.
• ArcFace / VISPR / inversion protocols not disclosed. The 0.99 → 0.01 ArcFace collapse (Appendix B), the 0.63 → 0.27 VISPR headline (Appendix L), and the 98.0% / 45.3% inversion experiment (Appendix B) are all central rhetorical points, yet none of the probe architectures, identity galleries, decoder architectures, or training protocols are specified. On VAD datasets that do not natively provide identity labels, how identity classes were constructed is a hard prerequisite for reproduction.
• Table 5 baselines are unhyperparameterized. Blur kernel size, DP noise scale / clip norm / budget, feature-bottleneck dimension, and GRL $\lambda$ are all unspecified. The "G-OPL beats these" comparison is therefore not repeatable in a like-with-like way.
• Attribute detectors (Tables 7–8). Named as "off-the-shelf" (gender), "HSV histogram thresholding" (clothing), "running as a proxy" (gait), with no thresholds, no localization method, and no map from binary signal to embedding $\mathbf{f}_{\text{attr}}^{(i)}$. The 73.3 → 49.5 multi-attribute leakage headline hinges on these detectors.
• Auxiliary experiments lack protocol. Clothing-color SSC = 0.84 (Appendix J), 14% rank-1 identity retrieval on blurred MSAD (Appendix L), and the "manually corrected" face-presence percentages in Table 15 either require released labels/models or a step (manual correction) that is intrinsically non-reproducible.
• Undefined external protocol. "MSAD Protocol ii" is referenced for the entire MSAD block (Tables 2, 3, 20–29) but not summarised in the paper.

Recommended Changes

Essential (each maps to a Weakness or Reproducibility item above)

• Soften absolute guarantees. Rewrite the "guarantees that downstream layers never observe identity-bearing directions" bullet in Appendix B (and equivalent phrasing elsewhere) to explicitly cover only the identified linear rank-k subspace, and reconcile with the non-zero FPD entries in Table 4 and the 45.3% face-recognition on inversions.
• Close the mean-embedding vs. identity gap. Either restrict the language to "face-related direction / faceness suppression" throughout, or add an experiment showing per-identity direction removal (e.g., variance retained per subject before vs. after projection). Also clarify that the ArcFace probe's input is the released residual $(\mathbf{I}-\mathbf{Q}\mathbf{Q}^\top)\mathbf{f}$ (fix or replace the $\hat{\mathbf{f}}^{(i)}$ notation which currently denotes the captured component).
• Reconcile Algorithm 1 vs. Algorithm 2. State explicitly whether G-OPL re-runs QR each step or relies on the soft penalty alone; if the latter, report empirical $\|\mathbf{Q}^\top \mathbf{Q} - \mathbf{I}_k\|_F$ during training and re-examine every "orthogonal projection" and "guarantee" claim in the G-OPL setting.
• Disentangle SSC / ARD / FPD in the discussion. Rewrite the "our method reduces identity leakage across (SSC/ARD/FPD)" sentence so SSC is described as capture (↑), ARD as utility preservation (↓), and only FPD/ArcFace as leakage (↓). Compare SSC only at matched k and add a random-Q null baseline.
• Fix the sub-chance FPD interpretation. Either restate what "accuracy" means for FPD (e.g., balanced accuracy or AP) so 0.01 is achievable, or diagnose the label / class-balance mismatch that produces sub-chance values, and update the metric definition in Sec. 3.3 accordingly.
• Repair the aggregate-performance captions. Change "consistently outperforming or matching top baselines" (Table 3) and "G1O0 consistently achieves top results" (Tables 22, 23, 26) to phrasing that matches what the Overall columns show ("competitive with, and in one configuration surpassing" / "best on Overall AUC while several deeper configurations exceed it on specific types").
• Publish the FPD / ArcFace / VISPR / inversion / attribute-detector protocols in an appendix (or as code) — probe architecture, split, class-balance handling, identity gallery construction, decoder architecture and training loss, gender classifier weights, HSV thresholds, gait proxy operationalisation.
• Publish the Table 5 baseline configurations (blur kernel, DP noise scale/clip/budget, bottleneck dim, GRL $\lambda$/schedule) — without these the "G-OPL is best" headline is not verifiable.
• Report per-seed variance on at least the Table 1, Table 4, and Table 6 headline cells (mean $\pm$ std over $\ge$ 3 seeds) and disclose optimizer, LR, schedule, and seed in Table 10.
• Reconcile table numbers. Investigate the Table 27 ↔ Table 28 Overall-AUC mismatches, the Table 28 CUHK AUC/AP transposition from G3O3 onward, the Table 20 (B1C1) vs (ALL) duplicated Vandalism/Water/Overall cells, and the VISPR 0.27 vs. Table 16 mean of ~0.33; publish corrected tables.

Suggested (map to more localized Weaknesses)

• Fix the sensitivity-range sentence. Change "improves as $k$ increases from 2 to 16, peaking around $k$ = 4" to "from 2 to 4" (Sec. 4.2, "Sensitivity to $k$..."), and restore the missing (i)/(ii) items before the orphan "(iii)" in the decoupled-rank paragraph.
• Anchor the multi-attribute progression consistently. Rephrase "AUC remains stable (85.3 → 84.4)" so it is anchored at the same row as the leakage progression 73.3 → 49.5 (either both from Baseline or both from Face), and reconcile the Table 7 baseline Face FPD (78.2) with the Table 8 baseline (73.3).
• Report the ShT/CUHK "+8.4/+3.5" as MGFN-specific. State the backbone next to these deltas so the reader does not read them as the RTFM effect (which is +0.5 on ShT).
• Update the practical k heuristic. Bring the "2–5% of feature dimension" recommendation into line with the paper's own optima at k = 4/8/16 on 1024/2048-d features.
• Bridge the MI < 0.02 / large-gain tension. Either add evidence for the specific model artefact by which a near-independent face channel drives large AUC changes, or moderate the "large AUC gains" language for ShT/CUHK.
• Address the multi-attribute construction. Discuss whether k should scale with attribute count, whether a block-diagonal $\mathbf{Q}$ is needed for near-orthogonal attributes, and how "concatenation" preserves $\mathbf{f}_{\text{attr}}^{(i)} \in \mathbb{R}^d$.
• Justify Ped2's late-stage exception. Either give evidence for the "low-res faces → better handled late-stage" mechanism or acknowledge Ped2 as an unexplained exception to the early-placement design principle.
• Temper the "primary practical privacy risk" framing. Cite the threat model that motivates it or acknowledge that this is an assumption of the deployment setting, not an established fact.
• Define "MSAD Protocol ii" in one or two sentences so the extensive MSAD ablations are self-contained.