Towards Hierarchy–Uniformity Equilibrium: Recovering Semantic Depth in Hypergraph Contrastive Learning

Summary

The paper takes aim at a genuinely interesting failure mode of hypergraph contrastive learning: standard InfoNCE-style objectives inherit hyperspherical uniformity, which treats every non-positive node pair as equally repulsive and — the authors argue — flattens the multi-level semantic structure that hyperedges naturally encode. They christen this the Hierarchy–Uniformity Conflict and its geometric consequence Semantic Flattening, then propose HyperDepth, which addresses both with a coordinated spectral + geometric design: a dual-band Chebyshev encoder with a learnable per-node gate to separate high- and low-frequency content, an energy-based hierarchical alignment module that binds nodes to a learnable prototype tree via a path-coherence energy trained with NCE-style path perturbations, and a standard InfoNCE branch acting in a dedicated projection space. The conceptual move — routing local discriminability through the projection head while keeping hierarchical alignment in the representation space, and formalizing an existence claim for a hierarchy–uniformity equilibrium under a frequency-separation assumption — is the paper's main contribution. Empirically, HyperDepth attains the best average rank across 15 datasets spanning homophilic and heterophilic regimes and 17 supervised / self-supervised baselines, with a full ablation showing that both the spectral and hierarchical components matter. The framing is clean, the writing is largely good, and the pieces do fit together. My principal concerns are that the theoretical statements are consistently narrower than what the surrounding prose claims (Assumption 1 essentially prescribes the orthogonality result; Proposition 1 drops the coherence coupling for leaves and the parent term in its upward recursion; the definition of the conflict itself overreaches for shared-ancestor pairs), the central phenomenon of `semantic flattening' is never measured on any baseline, no dedicated metric of hierarchical depth is reported, and reproducibility is thin — no released code, no seeds, no augmentation hyperparameters, no $\lambda_{\text{coh}}$, and no baseline tuning protocol.

Strengths

• Well-posed conceptual target. Framing the tension between uniformity and hierarchical semantics as a geometric conflict and giving it a name (Semantic Flattening) is a useful reframing that makes the design choices legible.
• Coordinated spectral + geometric design. The dual-band Chebyshev encoder with per-node gating, plus the decoupling of the local objective into a projection space while the global alignment stays in the representation space, is a clean architectural embodiment of the theoretical goal (routing high-frequency content to instance discrimination and low-frequency content to hierarchy).
• Broad and generally consistent empirical support. Across 9 homophilic + 6 heterophilic datasets and 17 baselines, HyperDepth achieves the best A.R.\ of 1.89 and is best on most datasets under a fixed linear-probing protocol. The full ablation in Table 6 shows that each component contributes, and the sensitivity sweeps over $\gamma$, $d$, $R$, $K$, $\tau_{\text{proto}}$ are reassuring — the method is not brittle to hyperparameter choices.
• Explicit theoretical scaffolding. The subspace decomposition (Theorem 1), the equilibrium existence claim (Theorem 2), and the prototype-hierarchy structural result (Proposition 1) tie the design to interpretable properties rather than leaving the paper purely empirical.

Weaknesses

• `Semantic Flattening' is postulated but never measured. The mechanism motivating the entire method is asserted geometrically but never demonstrated on existing baselines. There is no per-baseline diagnostic (e.g., intra- vs. inter-group distance ratios by hierarchical level, radial distribution, or intrinsic-dimension estimates) showing that TriCL / HyFi / HyperAim actually collapse hyperedge hierarchies. The reader has to take the failure mode on faith.
• Definition 1 overreaches for shared-ancestor pairs. $S_i$ is defined as any node sharing a semantic ancestor with $i$, yet $J_{\text{Global}}$ aligns each node to its own leaf prototype. Two members of $S_i$ that share only a mid-level ancestor are pulled to different leaves, not toward `coincidence' — so the definitional statement that maximizing $J_{\text{Global}}$ pulls all of $S_i$ toward coincidence does not hold in general.
• Proposition 1 is not proved as stated when $\lambda_{\text{coh}} > 0$. The leaf-alignment step optimizes only the instance–leaf term, ignoring the coherence coupling between the leaf and its parent that also depends on $c^{(K)}_\ell$. The internal-node recursion defines the neighborhood to include the parent but then propagates only descendant leaf means upward, silently dropping the parent contribution. Both steps must be tightened.
• NCE $\to$ energy reduction in Proposition 1 is unjustified. The proposition speaks of `minimizers of $L_{\text{Global}}$' but reasons purely about positive-path energy; the negative partition function from path perturbation is not accounted for.
• Theorem 2 conclusion is stronger than the hypothesis. The theorem allows $\tilde h^{\text{high}}_i$ to only approximately minimize $L_{\text{Local}}$ but concludes an equilibrium in the sense of Definition 2, which requires strict minimization. Remark 2 then amplifies this to `optimal with respect to both objectives'. The construction also silently assumes the sum $h^{\star,\text{low}}_i + \tilde h^{\text{high}}_i$ is a feasible unit-norm embedding — under the normalization the paper otherwise uses, the two subspaces share a norm budget and the independent minimizers need not superpose feasibly.
• `Orthogonal spectral components' is essentially Assumption 1. The abstract, intro and Section 6 present the orthogonality as a theoretical guarantee, but it is a direct consequence of a chain-rule identity under Assumption 1's clean separation of $g_{\text{loc}}$ and $g_{\text{sem}}$. The encoder + gating mechanism does not enforce Assumption 1 — the gated fusion mixes the two subspaces before either projection — so the assumption is a modeling idealization whose empirical validity is never assessed.
• Ablation attributions overstate what the theorems establish. Section 7 attributes the `equilibrium view' to Theorem 1 (it belongs to Theorem 2 / Definition 2) and cites Theorem 2 in support of the low-pass channel being more essential on homophilic graphs (Theorem 2 treats the channels symmetrically and says nothing about relative importance).
• Path perturbation is inconsistent with a strict tree. Replacing an arbitrary level-$k$ prototype and `preserving all other ancestors and descendants' is only well-defined if the prototype pools are level-wise sets, not a strict parent–child tree. The paper needs to pick one and rewrite the perturbation description accordingly.
• Missing baseline: no hierarchical-prototype contrastive method. The central claim is that prototype-hierarchy alignment recovers semantic depth, but no PCL/SwAV/HCL-style prototype baseline appears in Tables 1–2. The comparison thus cannot separate hierarchy helps' from HyperDepth's particular hierarchical implementation helps'.
• **No metric for semantic depth'.** The learned prototype tree is fully unsupervised, so recovering semantic depth' is defined only by the loss that shapes the tree. No independent hierarchical accuracy, hLCA, or taxonomic-consistency measurement is reported, making the central claim self-referential.
• Close margins reported without significance tests. Twitch (52.0 vs 51.5), Pokec (57.2 vs 57.0), Senate ablation and Amazon sit within one standard deviation; the `best average rank' framing would benefit from a Wilcoxon / Friedman test over per-seed ranks.
• Cora-A ablation asymmetry glossed over. On Cora-A, w/o $L_{\text{Local}}$ collapses by roughly 25 points (56.10 vs 80.71) while w/o $L_{\text{Global}}$ barely moves — that is not the `two branches remain competitive with each other' picture the narrative paints for the other homophilic datasets.
• No computational-cost comparison. The method adds per-node path search, periodic geometric rectification, and a second projection head. Only an asymptotic $O(K\bar B)$ figure is given; wall-clock and memory comparisons to TriCL / HyperAim / HyFi are absent.
• Notation and consistency slips. $h^{(1)}_i$ appears without a definition in the semantic-projection sentence; Table 3 conflates $P_i$ with $P_i^+$; the coefficient bound $\alpha_{uv} \ge 1$ is inconsistent with untraversed edges; the bolding convention in Tables 1–2 mixes column max' with self-supervised block max'.

Reproducibility & code

The `paper-code' evidence pass indicates no linked public code release, and the appendix leaves several load-bearing details unspecified. The main text and Appendix D give a coherent high-level recipe (ChebNetII backbone, dual-view augmentation, AdamW with cosine schedule, per-dataset $R$, $K$, $d$, $\gamma$, $\tau$, $k$ in Table 5), but the following gaps materially affect reproducibility:

• No source code or seeds. The 5-run means come with no seed list, no split-generation seed, no version pins, and no code release — several headline margins (Twitch, Pokec, Senate) are inside one standard deviation, so reproducing them from prose alone is unlikely.
• Contrastive augmentation hyperparameters missing. The paper describes feature masking + Gaussian noise as the view generator but does not report the masking rate, noise $\sigma$, or whether the two views use independently sampled corruptions. For InfoNCE-based methods these settings dominate the achievable accuracy.
• $\lambda_{\text{coh}}$ never given. The path-coherence weight is central to Eq.\ (5) and is required to be positive by the theoretical analysis, yet it appears in neither the main text nor Table 5. A per-dataset value (or the fixed default) needs to be documented.
• Projection-head architectures unspecified. $g_{\text{loc}}$ and $g_{\text{sem}}$ underpin Assumption 1 and thus the orthogonality result, but their depth, width, activation, and output dimensionality are not documented anywhere.
• **Structure' field in Table 5 undefined.** Each row lists e.g.\ Structure: [16, 32, 64]', presumably per-level prototype counts, but the paper never defines this list or its relationship to $K$ / the tree branching factor.
• Baseline tuning protocol not documented. For a claim of best average rank across 17 baselines, the paper does not state whether baseline hyperparameters were reused from prior papers, re-tuned on the shared validation splits, or copied from HyFi's numbers.
• Interaction between gradient-based and hard-EM prototype updates is unclear. Appendix D.2 says rectification freezes gradients on $C$; Algorithm 1 line 29 updates $C$ via AdamW every epoch. The intended two-mode schedule (gradient by default, hard overwrite every $T_{\text{rect}}$ epochs) is coherent but should be spelled out, including whether AdamW state is reset after overwrite and whether $z^{\text{sem}}$ is recomputed with the post-rectification prototypes.
• Gating behaviour under channel ablations is ambiguous. w/o low-pass' / w/o high-pass' leave the gating MLP undefined: is $\alpha_i$ frozen at 0/1, is the MLP still trained, or is the disabled branch simply zeroed pre-gate? Given the size of the Cora-A collapse under related ablations, this matters.

Recommended Changes

Essential

• Measure semantic flattening directly. Report on at least one dataset a per-hierarchy-level distance / density diagnostic (or an intrinsic-dimension estimate) comparing HyperDepth against strong contrastive baselines (TriCL, HyFi, HyperAim), so that the failure mode named `Semantic Flattening' is empirically demonstrated rather than posited.
• Add a hierarchical / prototype-based contrastive baseline. Include at least one PCL/SwAV/HCL-style baseline (adapted to hypergraphs) in Tables 1–2, so that the ablation-plus-comparison narrative distinguishes hierarchy helps' from HyperDepth's particular hierarchical design helps'.
• Introduce a dedicated hierarchical-depth metric. Report a hierarchical-accuracy / hLCA / taxonomic-consistency score on datasets that admit an external taxonomy (e.g., 20News, ModelNet40, citation networks), so the paper's core claim of `recovering semantic depth' is evaluated against something independent of the loss that shapes the tree.
• Repair the Proposition 1 proof. Add the coherence coupling to the leaf-alignment step (and either state that the alignment is with $\mu_\ell + \lambda_{\text{coh}}\alpha c_{\text{parent}}$ or explain why the parent contribution vanishes), and handle the parent term in the internal-node recursion rather than silently dropping it. Justify the NCE-to-positive-path-energy replacement (e.g., temperature limit or invariance argument) or restrict the proposition to `minimizers of the alignment energy'.
• Reconcile Theorem 2 with Definition 2. Either drop (approximately) minimizes' in the hypothesis and argue that the exact high-frequency minimizer exists, or weaken the conclusion (and Remark 2) to approximate equilibrium'. Address the unit-norm feasibility of $h^{\star,\text{low}}_i + \tilde h^{\text{high}}_i$ explicitly.
• Tighten Definition 1 for the shared-ancestor set. Either restrict $S_i$ to same-leaf siblings, or use a level-dependent target in $J_{\text{Global}}$ (e.g., alignment with the deepest common ancestor's prototype), so the `disjoint regimes' claim holds for the $S_i$ actually used.
• Reframe orthogonality as an assumption-driven consequence. In the abstract and Section 6, state clearly that `orthogonal spectral components' is a consequence of Assumption 1 rather than a derived property of the encoder, and add an empirical audit of how closely the trained encoder satisfies Assumption 1 (e.g., variance of $g_{\text{loc}}$ output projected onto the low-frequency subspace).
• Release code, configs, and seeds. Provide a public repository with training / evaluation scripts, per-dataset config files, exact random seeds for the 5 runs and the split generator, and the augmentation hyperparameters (masking rate, noise $\sigma$). This is a hard requirement for the `best average rank on 15 datasets' claim to be verifiable.
• Document the missing hyperparameters. Add $\lambda_{\text{coh}}$, the augmentation rates, the architectures of $g_{\text{loc}}$ and $g_{\text{sem}}$, and a definition of the `Structure' field to Table 5 / Appendix D.

Suggested

• Correct the theorem attributions in Section 7. Attribute the equilibrium view' to Theorem 2 / Definition 2 (not Theorem 1), soften agrees with Theorem 2' to `consistent with the low-frequency/global-semantics association from Section 6' when discussing channel ablations, and remove the implication that Theorem 2 predicts which channel dominates on homophilic graphs.
• Fix the path-perturbation description. Either state that the prototypes are level-wise pools (and drop the tree terminology), or restrict perturbation to preserve the ancestor/descendant chain and rewrite the `preserving all other ancestors and descendants' clause accordingly.
• Address the Cora-A asymmetry. Add a paragraph acknowledging that on Cora-A the local objective is dominant, and comment on what this implies for the `balanced complementarity' framing.
• Add significance testing. Report a Friedman/Wilcoxon (or paired seed-wise) test over per-dataset ranks for Tables 1–2, and mark headline margins that are inside one standard deviation.
• Report training cost. Add wall-clock time and peak memory for HyperDepth vs. TriCL / HyperAim / HyFi on at least one large dataset (e.g., 20News or Pokec), so the added cost of prototype path search + geometric rectification is auditable.
• Fix notation slips. Replace $h^{(1)}_i$ with $h_i$ in the semantic-projection sentence (or define $h^{(1)}_i$ explicitly), distinguish $P_i$ from $P_i^+$ in Table 3, correct $\alpha_{uv} \ge 1$ to $\alpha_{uv} \ge 0$, and state the bolding rule in the Tables 1–2 caption.
• Clarify the gating handling under channel ablations. State whether w/o low-pass' / w/o high-pass' freezes the gate, disables the gating MLP, or zeroes the removed branch before gating; report the corresponding accuracy consistently.