Scalable Event Cloud Network for Event-based Classification

Summary

SECNet proposes an "Event Cloud" network for event-based classification, action recognition, and human pose estimation. The paper's core conceptual move is to treat the raw event stream neither as a frame/voxel image nor as a permutation-invariant point cloud, but as a chronologically ordered 4-tuple sequence with polarity as a first-class structural coordinate; the network's Event-based Group and Sampling module then uses a learnable per-coordinate scaling (D-FPS), feature-space KNN grouping (EF-KNN), and a Coordinate Evolution Strategy to induce polarity- and time-aware neighbourhoods. A second, largely independent contribution replaces the per-group MLP with a Fourier-domain Hadamard product (Spatial-FA and Temporal-FA), which the authors argue extracts long-range spatio-temporal structure at $O(D_i \log_2 D_i)$ rather than $O(D_i^2)$ cost. Across ten benchmarks the paper reports strong accuracy at markedly lower parameter and MAC budgets than voxel and frame baselines, with FPGA measurements as an additional efficiency argument.

The intuition — that polarity carries information best modelled structurally, and that a length-$T$ spectrum is a cheap surrogate for long temporal receptive fields — is a plausible and useful contribution, and a from-scratch reimplementation confirms the architectural skeleton is internally consistent (the 64/132/268/540 hierarchy, the axes on which each FFT is taken, the ~19.5× Spatial-FA MAC reduction against a matched conv baseline). The main conceptual limitation is that two of the paper's most important selling points are empirically underspecified. First, the Spatial-FA ablation (Table 13) shows that a conventional convolution matches or beats SFA on accuracy across most datasets, so SFA is an efficiency substitution, not a joint efficiency-and-accuracy win. Second, the flagship "16% over previous SOTA" gain on UCF101-DVS is measured against VMST-Net, while the same paper's Table 6 lists EventMamba only ~1.3 points behind SECNet — the true delta over the strongest baseline is small. Compounding these issues, the released repository ships no implementation at all (only a README that references scripts and preprocessing entry points that do not exist), so every headline number requires an independent reimplementation before it can be re-run, and where such a reimplementation was possible on the two obtainable datasets it landed 15+ points below the paper on N-Caltech101, ~5× below on throughput, and ~2× above on GMACs. The architectural contribution is nonetheless real and the relative efficiency structure meaningful when read as such.

Strengths

• Representation-level contribution. Preserving chronology while promoting polarity to a coordinate that participates in D-FPS, EF-KNN, and CES is a genuinely different design choice from prior point-based event work, and one that can be described independently of the frequency modules. A reimplementation-side check confirms Event Cloud is order-sensitive: shuffling events in time drops N-MNIST accuracy from 0.9939 to 0.9843, supporting the paper's premise that chronological ordering is a load-bearing property of the representation.
• Efficiency argument, correctly framed. The Spatial-FA analytic reduction from $O(D_i^2)$ to $O(D_i \log_2 D_i)$ (~60× per element at $D_i=540$) is derivable on paper, and the ~19.5× network-level SFA MAC reduction between matched SECNet and ECNet builds is reproduced almost exactly (19.504× on every profiled dataset; absolute Spatial-FA MMACs within ~3.5% of the paper). Taken as a relative efficiency claim rather than an absolute accuracy claim, this is the paper's most solid result.
• Structural / architectural consistency. The 64 → 132 → 268 → 540 feature dimension progression generated by the paper's prev × 2 + 4 rule, the three-stage coordinate halving, and the choice of FFT axis (channel dim for SFA, event dim for TFA) all reproduce cleanly in an independent implementation, and Figure 6's assertion that "the light blue is congruent" (the non-SFA remainder is byte-identical between SECNet and ECNet) holds in the reimplementation to the last decimal.
• Breadth of evaluation. Ten datasets across three tasks (object, action, pose) is substantially wider than most point-based event papers, and the inclusion of a high-resolution (1280×800) THU-E-ACT-50 benchmark supports the scalability narrative.
• Preprocessing-time argument, in its relative form. Where a per-representation comparison could be re-run (Voxel Grid vs Event Cloud on N-Caltech101), the ~9× ratio matches the paper's 9.7×; the ordering of the whole preprocessing table did not reproduce (Time Surface came in faster than Event Cloud, and Binary Map was only ~12× slower rather than ~500×), so the Voxel-vs-EC efficiency argument holds where the wider table's ordering does not.

Weaknesses

• UCF101-DVS "16% over previous SOTA" is measured against a non-strongest baseline. Table 6 lists EventMamba at 0.903 vs SECNet 0.916 — a ~1.3-point delta, or ~1.4% relative. Choosing VMST-Net (0.787) as "previous SOTA" produces the headline 16%, but that comparator is not the strongest one in the paper's own table. The Section 4.4 conclusion that voxel/frame representations irrecoverably discard information does not follow from the actual best-baseline gap; the same check on DVSGesture (0.989 vs 0.992) and DailyDVS (0.9965 vs 0.996) also flips direction.
• Ablation of Spatial-FA contradicts the "indispensable components" framing. Table 13 shows ECNet (SFA → conv) matches or beats SECNet on DHP19, ASL-DVS, UCF101-DVS, and N-Cars. On UCF101-DVS ECNet even reaches 0.941, above what the main text calls "previous SOTA VMST-Net" (0.787). This directly undermines Section 4.6.1's "all designed components are indispensable" and the framing of SFA as a lossless efficiency substitute — the true SFA story is efficiency at a small accuracy cost.
• Efficiency headline for THU-E-ACT-50 quotes numbers that do not match any SECNet configuration in the paper. The Section 4.4 sentence "1.27 M / 0.971 MACs" does not match any SECNet row in Table 13; 0.971 GMACs is exactly the ECNet N-Cars value (convolution baseline), and 1.27 M does not match Table 12's 1.103 M for the corresponding centroid configuration. Either the flagship comparison against EV-ACT accidentally cites the convolution variant, or a THU-E-ACT-50-specific SECNet configuration is being cited without documenting it — as written this is a serious internal inconsistency.
• Object-classification comparison omits the strong point-based baselines the paper itself uses elsewhere. Table 2 benchmarks against only EventNet (2019) and PointNet++ (2017) among point methods; TTPOINT, SpikePoint, and EventMamba appear on other tasks in the same paper. The "27% over SOTA Point Cloud" statement is therefore against a dated comparator.
• No per-seed variance. All ten datasets and all ablation rows are single point estimates. Several decisive comparisons (DVSGesture 0.989 vs 0.992, ASL-DVS 0.9993 vs 0.997, N-Cars 0.947 vs 0.958) sit inside the range of training-run noise typical for point-based event models.
• Polarity-integration ablation scope. The Input-P / w/o-P comparison is run only on datasets with meaningful ON/OFF asymmetry (DHP19, N-Cars, UCF101-DVS); saccade-generated datasets where polarity is nearly random (N-MNIST, N-Caltech101) — the ones that dominate Table 2 — are not tested. A reimplementation-side check on N-Caltech101 does show structural polarity helping, but the arms are epoch-unequal so this is suggestive rather than diagnostic.
• Method-side under-specification. EF-KNN grouping in feature space is not paired with a specification of the centroid feature $F_{C_i}$ (the centroid comes from coordinate D-FPS but the neighbourhood is in feature space, so the mapping must be stated); several Section 3 display equations are missing; AGG is described as aggregating "on the $t$ dimension" while collapsing $K$; the sentence "SECNet replaces LSTM for temporal feature extraction, which makes it difficult to capture long-term dependencies" inverts the intended meaning.
• Numerical / notation inconsistencies. The SFA speedup is quoted as $10^4$, 60×, 20×, 5×, and 40× in different sections; "2% of Point Transformer" is arithmetically ~3%; 229.7 FPS (appendix) vs 120 FPS (Table 6) is left unlabelled; "5%" (Table 5) and "16%" (UCF101-DVS) mix relative and percentage-point conventions; $F_{Si}$, $F_{Gi}$, and $F_{Ai}$ are used with three different dimension orderings; "84 s" for RAMB18 block counts mixes units.
• Overreach on polarity and prior work. The absolute claim that Point Cloud methods "ignore polarity" contradicts the Introduction and Table 11's own taxonomy; the Appendix K statement that "Previous works have treated the events' $t$ coordinate as the space $z$ coordinate" is contradicted by the paper's own citations to EventLSTM, TTPOINT, SpikePoint, and EventMamba.
• Preprocessing-time table lacks controls. Table 4 leaves per-representation parameterization, thread count, and CPU model unstated; independent measurement on N-Caltech101 fails to reproduce Event Cloud as the fastest representation (Time Surface beats it) and cannot reproduce the ~500× Binary Map penalty. Only the Voxel/EC ~10× ratio reproduces.
• Impact statement is generic. The paper's strongest gains are on action-recognition datasets, and Section 4.7 argues for hundreds-of-FPS edge deployment; the impact statement should engage the specific real-time gesture / action classification surveillance scenario rather than waving away privacy risk with "no additional risk beyond existing event-camera technologies".

Reproducibility & code

• No SECNet model, training, or dataloader code was released. The abstract's code-availability URL points to a repository that, at the time of review, contained exactly six files (README.md, SECNet.png, and four demo GIFs) across every branch and every commit. The README documents a workflow (cd dataprocess; python generate_xxx.py, python train_frequency.py) whose entry points do not exist. Every headline number (Tables 2, 3, 5, 6, 7, 13) therefore required a full independent reimplementation on the authors' earlier EventMamba scaffold. This is the central reproducibility finding of the review — everything below is downstream of it.
• Two critical architectural details are unspecified. (a) The Spatial-FA / Temporal-FA learnable filter shapes are never given in prose — the paper only says "dimensionality matching that of the spectrum". Recovery required inverting Table 12's per-configuration parameter counts across four centroid tuples; the only shapes consistent with those counts are complex $(K, D_i/2+1)$ for SFA and complex $(T_i/2+1, D_i)$ for TFA. Parameter agreement with Table 12 is therefore fitted, not independent. (b) The learnable $\alpha$ in D-FPS has no defined gradient path — a literal reading of Eq. 8 gives $\partial\mathcal{L}/\partial\alpha \equiv 0$ because FPS is a non-differentiable index selection, which would make Table 8's "w/o $\alpha$" row vacuous.
• MAC accounting convention is unstated. thop and fvcore score torch.fft.rfft as zero MACs, so every GMACs figure in the paper depends on a counting convention that is nowhere stated (real vs complex, iFFT included, Hermitian symmetry, grouping/KNN, etc.). An analytic counter written from scratch, reverse-engineered from the paper's own "$540 \to$ factor of 60" arithmetic, gives GMACs 1.9–2.1× the paper's numbers on every configuration (e.g. DHP19 pose 0.333 G vs 0.160 G, UCF101-DVS 0.902 G vs 0.48 G). Absolute GMACs are unverifiable by construction until the convention is stated; the relative Spatial-FA vs conv ratio (~19.5×) is robust across conventions.
• Throughput does not reproduce. Measured on an NVIDIA A100 80 GB PCIe at the UCF101-DVS configuration, the reimplementation delivers 43.43 FPS at 23.03 ms/sample — 5.3× below Appendix G's 229.7 FPS and 2.8× below Table 6's 120 FPS. A full 20-cell sweep of centroid and event-count combinations peaks at 173 FPS on the smallest configuration; no cell reaches the 200 FPS "guarantee" for the two rightmost centroid columns of Table 12. Some of this is hardware (A100 vs RTX 4090 at batch 1), but not the whole gap.
• Headline accuracy partially reproduces where the data is obtainable. N-MNIST at 94/150 epochs reaches 0.9943 vs the paper's 0.997 — well within a reasonable band. N-Caltech101 at 26/150 epochs reaches 0.6681 vs 0.824, a 15.6-point gap; three orthogonal issues make the gap hard to diagnose cleanly — undertraining (wall-clock truncation), an unstated split policy (window-level vs recording-level, where naive window-level splitting leaks recordings), and a sample-count mismatch (21,545 vs 26,122). The other three object-classification datasets, all four action datasets, and DHP19 were unobtainable (see next bullet).
• Data availability is a documentation gap. Four datasets in the ten-dataset suite are access-gated behind per-user portals (N-Cars: Prophesee licence; DHP19: authors' portal; DVS128 Gesture: IBM registration; THU-E-ACT-50: author application), and three further ones are dead third-party download endpoints (CIFAR10-DVS figshare returns HTTP 202; ASL-DVS Dropbox md5 fails; UCF101-DVS is a Google Drive archive behind interactive auth). Neither the paper nor the README notes that most of the evaluation suite requires per-dataset registration, and no data-access appendix exists.
• UCF101-DVS filename split is deferred to prior work and not shipped. Table 6's 0.916 relies on the "filename-based partitioning method (Ren et al., 2025)", neither described in the manuscript nor included in the released materials — even given the raw data, the specific split is not recoverable.
• DHP19 pose head is described only at Algorithm 1 summary level. No 2D→3D lifting details, camera-pair usage, MPJPE reference frame, or subject/camera split are given, and the pose regressor is not in the released code. The reimplementation therefore could not profile even the correct pose-head parameter/GMACs numbers — Table 7's 1.831 M / 0.160 GMACs is architecture-only under an inferred head.
• Table 10 FPGA numbers are unverifiable and internally implausible. No HDL sources, testbenches, or synthesis reports were released ("relevant code will be released upon completion of this work"). Independently: Table 10 reports the same 393,859 cycles for SFA at three different transform lengths (64/128/256), which an $O(D \log D)$ FFT running at a fixed clock cannot produce, and the paper offers no explanation (e.g. fixed-latency pipeline with padding).
• Ablation study is not runnable from the released repo. The seven Table 8 rows (w all, w/o $\alpha$, w/o EF-KNN, w/o SFA, w/o TFA, Input P, w/o P) are well-defined but require code toggles that do not exist. A substitute run on N-MNIST / N-Caltech101 (the only obtainable datasets) preserves the qualitative direction — ablated variants score below the full model — but the ordering the paper reports is contradicted: removing SFA is the least damaging ablation on N-MNIST, not the most. This may be a training-length artefact or a dataset-conditional effect; without the paper's own datasets it cannot be resolved either way.
• Training config disclosure is insufficient. Batch size is stated as "dynamically set", and random seeds, weight-decay, augmentation policy, gradient clipping, and $\alpha$ initialization are unstated. Given how narrow the gaps to competing baselines are, a reproducer cannot distinguish a real gap from a config difference.
• Section 3 display equations are missing. Several sentences ("which can be defined:", "Event Cloud can be generated by:", "the overall computational flow of the SECNet pipeline is formulated as follows:") lead into equations that do not render in the manuscript source. Combined with the missing code, the pipeline recurrence and DFT step cannot be cross-checked against the intended formalism.

Recommended Changes

Essential

• Release the actual model, training, evaluation, and preprocessing code, plus at least one checkpoint per benchmark. Include the UCF101-DVS filename-based split, the DHP19 pose head with 2D→3D lifting, and toggles for every Table 8 ablation. This is the single highest-priority change — without it, no headline number in Tables 2, 3, 5, 6, 7, or 13 is independently checkable.
• Add prose specifications for the two critical architectural gaps. In Section 3.2.2 give the exact SFA and TFA learnable-filter shapes (real vs complex, and the exact axes); in Section 3.2.1 explain how $\alpha$ receives gradient given that FPS is a non-differentiable index selection, and state its initialization.
• State the MAC accounting convention explicitly — whether real vs complex multiplies, iFFT, complex-Hadamard, Hermitian symmetry, and grouping/KNN are counted — and reconcile the 1.9–2.1× discrepancy between the paper's reported GMACs and a straightforward analytic FFT counter.
• Reconcile the Spatial-FA ablation with the framing. In Section 4.6.1 and the abstract, state plainly that Spatial-FA is an efficiency substitution that trades a small accuracy cost for a large MAC reduction, and remove the "all designed components are indispensable" claim. Report the accuracy delta explicitly for the Table 13 datasets where ECNet is stronger.
• Recompute the UCF101-DVS delta against the actual strongest baseline (EventMamba, 0.903) rather than VMST-Net (0.787), and revise the Section 4.4 conclusion that voxel/frame representations "cannot recover" the omitted information. Do the analogous check on DVSGesture and DailyDVS.
• Fix or clarify the "1.27 M / 0.971 MACs" comparison against EV-ACT. In Section 4.4 either substitute the correct SECNet THU-E-ACT-50 numbers or, if 0.971 MACs is the intended figure, tabulate the THU-E-ACT-50 configuration that produces it and reconcile with Table 12's 1.103 M.
• Add per-seed variance (mean ± std over ≥3 seeds) for Tables 2, 3, and 8, so that the sub-percentage-point gaps to EventMamba, EDGCN, and VMV-GCN can be judged.
• Expand Table 2's point-based baselines. Include TTPOINT / SpikePoint / EventMamba on N-MNIST / N-Cars / CIFAR10-DVS / N-Caltech101 / ASL-DVS (or state clearly that they were not re-run), and revise the "27% over Point Cloud SOTA" statement accordingly.
• Add a data-access appendix and per-dataset split policy. For each of the ten datasets, state the exact download URL/portal the authors used, whether registration or licence is required, and for CIFAR10-DVS / ASL-DVS / DailyDVS / UCF101-DVS / N-Caltech101 how the train/test split was constructed (window-level vs recording-level, sliding-window overlap policy).

Suggested

• Reconcile SFA speedup numbers into one derivation. Currently $10^4$ / 60× / 20× / 5× / 40× coexist; correct "2% of Point Transformer" to ~3%; annotate 229.7 FPS and 120 FPS with their (dataset, event count, centroid tuple) configurations.
• Standardize relative vs percentage-point language throughout Section 4 (Table 5's 5%, Table 6's 16%, Section 4.3's 27% currently mix conventions).
• Restore the missing Section 3 display equations for $E$, $EC$, the pipeline recurrence, and the DFT step; unify $F_{Si}$ / $F_{Gi}$ / $F_{Ai}$ naming and pick a single dimension ordering for the SFA input.
• Define EF-KNN more tightly. Specify how the centroid feature $F_{C_i}$ is constructed (D-FPS-parent feature, learned projection, or aggregation) and which distance metric is used (L2 vs cosine).
• Add a polarity ablation on saccade-generated data (N-MNIST or N-Caltech101), so the "structural polarity integration always helps" claim is tested on datasets where ON/OFF asymmetry is minimal.
• Report learned $\alpha$ values across datasets — a small table showing e.g. $\alpha_t$ vs $\alpha_{xy}$ per dataset would substantiate the D-FPS mechanism story.
• Label FPGA Table 10 as "to be released" and explain the constant SFA cycle count across three transform lengths (which is inconsistent with an $O(D \log D)$ FFT under a fixed clock); separate FPGA claims from software benchmarks visually. Fix "84 s to 39" to "84 to 39 RAMB18 blocks".
• Soften overreaching prior-work statements — the "Point Cloud ignores polarity" and "Previous works have treated $t$ as $z$" claims are contradicted by the paper's own citations and by Table 11.
• Fix the "SECNet replaces LSTM" grammar in Section 4.6.2 so it clearly refers to the SECNet-w-LSTM ablation, not SECNet itself; rewrite the AGG description so it clearly says the attention collapses the $K$ neighbour dimension.
• Tighten the impact statement to engage the specific action-recognition surveillance scenario (real-time gesture / action classification at hundreds of FPS on edge hardware), rather than a generic "no additional risk" line.