Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks

Summary

This paper develops order-equivariant neural networks (OENNs) as a unifying framework for equivariant deep learning on data indexed by finite posets, with the goal of subsuming permutation-equivariant models (DeepSets, MPNNs) and sheaf-based architectures under a single formalism. The setting is a finite poset $(P, \leq)$ carrying a finite group $G \leq \mathrm{Aut}(P, \leq)$ that acts on an equivariant feature bundle $(V_p, T^V_{\gamma,p})$. The paper (i) characterizes all linear order-equivariant maps by a transporter law and an orbit parametrization keyed by pair-orbits, (ii) turns this linear theory into a nonlinear architecture — orbital affine layers, equivariant biases, pointwise Reynolds blocks, and pair-orbit aggregation — and (iii) proves universal approximation theorems in three regimes: a full-OENN UAT via a broadcast+pointwise decomposition, and a diameter-sharp local UAT for the pair-state class that remembers a source label along a $G$-invariant communication relation $R$. Sections 4.1–4.2 recover fixed-graph MPNNs, circular convolutions, dihedral distance-isotropic filters, and cellular/simplicial sheaf layers as tied special cases of the orbit-parametrized construction. An appendix embeds OENN into category-equivariant networks (CENN) via the action groupoid $G \ltimes P$, notes that the thin poset category alone does not encode the OENN symmetry, and reports a single EMG-IMU multimodal-classification experiment for Grothendieck-CENN. The conceptual move — organizing equivariance around pair-orbits of a poset action groupoid, and identifying pair-state locality as the minimal source-aware mechanism restoring universality for local architectures — is genuinely useful and clarifies why anonymous message passing is not universal on face-poset data. The main limitations are conceptual overreach relative to the formal object (the recurrent "non-invertible / higher-order" framing sits with CENN, not OENN, which is entirely group-based), a heavy reliance on parallel Maruyama papers to carry empirical and comparative weight, and the absence of any experiment on the graph or sheaf side that the framework is explicitly built to unify. Several technical derivations are also compressed to a point where reproduction requires filling in nontrivial invariance conditions.

Strengths

• Conceptual contribution. The orbit parametrization plus transporter law gives a clean, uniform account of what a linear equivariant map on poset-indexed data is, and cleanly recovers block-tying rules of permutation-equivariant networks as a special case.
• Locality vs. universality separation. The distinction between anonymous relation-message-passing OENNs and pair-state OENNs, together with the diameter-depth compilation via source-aware pair-states, is a sharp and useful theoretical statement that isolates where ordinary MPNNs fail to be universal.
• Coverage. The framework meaningfully recovers DeepSets, fixed-graph MPNNs, circular/dihedral convolutions on cycles, incidence updates, and cellular/simplicial sheaf layers, and does so through orbit types rather than architecture-specific tricks.
• Action-groupoid embedding. The appendix's identification of completed OENN maps with CENN natural transformations over $G \ltimes P$ (rather than the thin poset category) is a substantive clarification, and the accompanying activation-hypothesis comparison (globally Lipschitz for the CENN route vs. continuous non-polynomial for the OENN route) is a useful technical caveat.
• Sheaf treatment. The invariant-metric adjoint construction of Lemma 4.1 and the pair-orbit sheaf aggregation are cleanly organized and give a genuine categorical-flavored account of orientation-aware sheaf layers.

Weaknesses

• Non-invertible symmetry claim vs. group-based formalism. The abstract and introduction repeatedly gesture at "non-invertible or higher-order" symmetries, but the OENN setup is entirely a finite-group action by order automorphisms, i.e. invertible. "$\mathrm{Aut}(P, \leq)$ involving non-invertible relations" is a category error; that structure belongs to CENN via the action groupoid or monoid actions and should not be attributed to OENN.
• Unsupported priority claim on sheaf UATs. The abstract's assertion that no UAT was known for sheaf neural networks is a blanket negative existential and is not defended with a survey or citation trail; given the theoretical activity around neural sheaf diffusion and Laplacian sheaf models since 2020, this needs either delimitation of scope or explicit substantiation.
• Empirical scope mismatch. The paper's central unification claim is about graphs and sheaves (Section 4), but the only experiment (Table 2) evaluates the CENN generalization on a multimodal EMG-IMU task and does not test any OENN specialization on a graph or sheaf benchmark. The graph/sheaf side of the framework is illustrated only by tied-parameter special cases, not by any demonstration that pair-state OENNs or orbit-tied sheaf layers beat existing baselines.
• Overreliance on parallel self-citations. Substantive empirical validation is deferred to a cluster of Maruyama (2025b;c; 2026a;b) and Maruyama & Yasuda (2026) papers, several of which are unpublished/in-press. A reviewer cannot cross-check these from the present submission, and the paper's overview paragraphs ("This yields improved generalization") state their conclusions as if they were established in this work.
• Miscellaneous mis-attribution. DeepSets is grouped with GNNs as achieving equivariance "with respect to the automorphism group of a graph"; DeepSets has no graph structure and is $S_n$-equivariant.
• Compressed technical steps. Several derivations rely on tacit conditions that should be explicit: the coarsened relation summary $S_R$ requires a single encoder simultaneously invariant under all $H_O$ with $O \subseteq R$; the "intertwiner for the target fiber representation" needs to intertwine two representations, not one; the dihedral distance-isotropic constraint $A_\ell = A_{-\ell}$ is a bare-matrix equality only in the permutation-only case; the pair-state "exact" diameter-$D$ realization glosses over how shorter-than-$D$ paths coexist with the rescaling $S$; and the off-diagonal lift in the proof of Proposition 2.16 asserts identity-value transport $z^1_{q,p} = h_p$ without checking that the transport factor is identity.
• Corollary A.7 activation-hypothesis conflation. The corollary states two density claims but presents them as one; the CENN-route requires globally Lipschitz $\alpha$, while the weaker "continuous non-polynomial" hypothesis comes only from the independent Section 3 argument. As written, a reader could over-generalize.
• Sharpness of the diameter bound. The "diameter-sharp" language is a pointwise existence claim on a worst-case pair, not a uniform lower bound on required depth for any target map; the current phrasing risks overreading.
• Norm/constant bookkeeping in Lemma 3.2. The Reynolds-average bound mixes $\|\cdot\|_*$ with the $H$-averaged $\|\cdot\|_0$, and the constants $c$ and $C$ interact without explicit tracking; the crux of the equivariant density argument would benefit from an explicit norm-conversion line.

Reproducibility & code

The submission does not release code, config files, or a reproducibility checklist for either the framework or the Table 2 experiment. The veritas evidence assesses the empirical claims read from Appendix A.3 and finds no methodology to support them. Nothing has been executed here — the assessment concerns what the released material does and does not cover.

• Headline accuracy and F1 numbers. The reported values in Table 2 (accuracies 0.428 / 0.646 / 0.904 and F1 0.399 / 0.592 / 0.903) are not accompanied by any architectural specification for TemporalCNN, InvariantTransformer, or GrothendieckCENN, nor by training hyperparameters (optimizer, learning rate, batch size, epochs, seeds), preprocessing (window/stride, normalization), split protocol (subject-level or otherwise), or repeat-run statistics. A reproducer would have to invent every architecture and training decision. The gap is large enough that the qualitative ranking may survive, but the specific values are not credibly reproducible.
• Baseline architectures are generic labels. "TemporalCNN" and "InvariantTransformer" are not defined; both terms cover a wide range of concrete implementations that will produce substantially different accuracies. Whether GrothendieckCENN retains its lead under matched-tuning strong baselines is not shown.
• Dataset ambiguity. The paper cites "Khan et al., 2020a/b" but the references list two related but distinct releases (Data-in-Brief sEMG dataset and the Mendeley Physical Action Dataset V2). The exact benchmark is not disambiguated, and no DOI, class taxonomy, or preprocessing is stated.
• F1 averaging convention unstated. F1 macro/micro/weighted meaningfully shifts the number on any class-imbalanced action-recognition task; the paper does not specify.
• No self-contained empirical support for the framework's main claims. Beyond the single EMG-IMU table, the paper points to five Maruyama 2025-2026 self-citations for further experimental evidence, none of which is bundled with this submission.

Recommended Changes

Essential

• Reconcile the non-invertible symmetry framing with the OENN formalism. Rewrite the abstract and Introduction so that "non-invertible / higher-order" is scoped to CENN via the action groupoid or monoid actions, and OENN is described as group-equivariant with respect to $\mathrm{Aut}(P, \leq)$ (or a subgroup). This resolves the category error in the "automorphism group ... may involve ... non-invertible relations" sentence.
• Support or soften the "no UAT known before" claim. Either survey the prior neural-sheaf theory (Hansen–Gebhart, Bodnar et al., Barbero et al., Bamberger et al.) and state precisely which UAT gap the paper fills, or replace the blanket negative with "we are not aware of a comparable UAT."
• Fully specify or move the Table 2 experiment. Add architectures for TemporalCNN / InvariantTransformer / GrothendieckCENN, hyperparameters, seeds and per-seed variance, subject-level splits, preprocessing, and the F1 averaging convention; disambiguate the dataset (with DOI). If these details cannot fit here, defer the empirical claim to the follow-up paper and remove the categorical "substantially outperformed" phrasing from the appendix.
• Release reference code. Provide a minimal implementation of the OENN primitives (orbital affine, Reynolds block, pair-orbit aggregation, sheaf-type layer) and the appendix experiment. The paper's central practical selling point is a framework, and adoption depends on a usable reference.
• Add an OENN experiment on the graph or sheaf side. Include at least one benchmark where the pair-state or orbit-tied OENN class is shown to outperform an anonymous-MPNN or standard sheaf baseline on a task the framework predicts will benefit — this is essential given Section 4 is entirely theoretical.

Suggested

• Fix the Table 2 caption to mention InvariantTransformer alongside TemporalCNN so it matches the models actually compared.
• Correct the DeepSets attribution in the Introduction (DeepSets is $S_n$-equivariant, not equivariant to any graph automorphism group).
• Separate the two density statements in Corollary A.7 so each is explicitly tied to its own activation hypothesis (globally Lipschitz for the CENN-route derivation, continuous non-polynomial for the direct Section 3 route).
• State the joint-stabilizer-invariance condition for $S_R$ explicitly where the coarsened relation summary is introduced; note that a tied encoder must be invariant under the subgroup generated by all $H_O$ with $O \subseteq R$.
• Fill in the omitted invariance conditions in the sheaf sufficient-condition derivation ("intertwiner for the target fiber representation" needs the input and output stabilizer representations named) and in the Lemma A.6 projection step (require $G$-stability of the sub-slots being projected onto).
• Add a construction sketch for the diameter-$D$ pair-state realization that explicitly handles paths of length $< D$ (self-loop padding vs. the walk-count factor $a_{q,p}$) so "exactly realizable" can be read without reconstruction by the reader.
• Tighten the constants in Lemma 3.2 by tracking which norm each bound lives in and how $c$ converts between $\|\cdot\|_*$ and $\|\cdot\|_0$.
• Clarify the dihedral filter constraint. State that $A_\ell = A_{-\ell}$ is the permutation-only-case form of a more general intertwining relation under a non-trivial dihedral fiber action.
• Soften the sharpness language for the diameter lower bound to make explicit it is worst-case pointwise, not a uniform depth requirement for arbitrary target maps.
• Rephrase "This yields improved generalization" in the Grothendieck-CENN motivation as an observation reported in the cited follow-up work, not a demonstrated fact of this paper.