Solving Time-Dependent Differential Equations with Physical Dynamical Systems

Summary

The paper proposes DS-TS, a Dynamical System Machine (DSM)–based analog solver for time-dependent PDEs. The core conceptual move is to stop treating a DSM as an equilibrium engine and instead have it realize the target PDE's trajectory in physical, continuous time, replacing the discrete-time step of classical and ML solvers with the intrinsic RC evolution of an electronic network. Three additions give the DSM enough expressiveness for realistic PDEs: an Excitatory–Inhibitory Inspired Coupling (EIC) that augments a learned linear mixing with sign-gated push–pull currents; a State-aware Dynamic Nonlinearity (SDN) that produces polynomial coefficients as functions of the instantaneous node state via a translinear log/exp pipeline; and a Hierarchical Temporal Integration (HTI) that stacks $G_m$-C integrators so a single node can carry high-order time derivatives. Boundary conditions are enforced by tying node dynamics rather than by loss penalties. On five 1D/2D PDE benchmarks and three grid resolutions, Table 1 reports MAE for DS-TS 1–3 orders of magnitude below PINN/FNO/UNet baselines and modestly below prior DSMs, with ~10^3× speedup and ~10^5× energy savings claimed against ML solvers on an A100, plus modest MAE degradation under a 52 dB noise model. The direction is compelling and the combination of E–I inspired coupling with dynamically generated polynomial coefficients is a meaningful step beyond first-order, static-coupling DSMs. However, several central claims outrun the reported evidence: the ~10^5× energy figure does not hold against the paper's own DSM baselines, which draw comparable or lower power at the reference scale; the three-component design has no ablation, so the contribution of each innovation is unattributed; the "ground truth" used for training and MAE is itself explicit forward Euler; Table 1 shows single-run "best" values with no seeds or variance; and the analog implementation description leaves sign handling in the log-domain polynomial and the relationship between the two-matrix EIC and the single-input sign gating unresolved. A from-text reconstruction of the method further underscores the empirical fragility: with ~21 specification ambiguities filled by assumption, the reconstructed DS-TS attained the strictly smallest MAE in only 2 of 14 head-to-head cells, and matched rather than dominated the DSM baselines. The mechanism is worth publishing, but the empirical and hardware-implementation stories need substantial tightening.

Strengths

• Conceptual contribution. Reframing DSMs from equilibrium finders to trajectory generators for TDDEs is a clean and well-motivated shift, and the mapping between the physical continuous-time evolution and the target PDE gives the paper a natural story about why the analog substrate should help.
• Component design. EIC (push–pull sign-gated currents), SDN (state-dependent polynomial coefficients via a translinear pipeline), and HTI (a cascade of $G_m$-C integrators for high-order time derivatives) each address a specific and clearly stated limitation of prior DSMs, and each has a plausible circuit-level realization.
• Boundary handling. Treating boundary conditions as intrinsic node dynamics rather than as auxiliary penalty terms is a nice contrast to PINN-style constraints, and the periodic and time-dependent BC formulations are stated cleanly in prose. A reconstruction shows the resulting periodic-consistency surrogate stays inside its reference across a 500-step rollout in both a 1D order-1 case and a 2D order-4 case, so the mechanism does behave as advertised.
• Benchmark breadth (within scope). The five PDEs span parabolic, hyperbolic, first-order, and fourth-order-in-time settings, and three grid resolutions are covered. Within the 1D/2D regime that the paper studies, the coverage is reasonable.
• Noise modelling attempt. Including a noise model that combines resistor and MOS thermal noise, and reporting a bounded MAE degradation, is more than most DSM/analog-computing papers offer. A reconstruction confirms that at 52 dB current-domain SNR the mean noisy/noiseless MAE ratio is ~1.001 across all 30 configurations — an even tighter form of the paper's "less than 2×" statement, and evidence that the rollout is dominated by discretization rather than by injected noise.
• HTI stability on high-order-in-time equations. With the HTI depth set as the paper prescribes, the reconstruction produces finite, smoothly growing 500-step rollouts on both the 2nd-order Wave and the 4th-order Hyperbolic equations, while neural baselines (UNet AR-1, FNO AR-1) diverge to $\sim 10^{31}$/$10^{26}$ on those same benchmarks — the "high-order-in-time solver" half of the contribution is real.

Weaknesses

• Energy overclaim. The abstract, contributions, and conclusion all state ~10^5× energy efficiency "compared to existing solvers," but Section 4.1 gives DS-TS at 1.12 W for 2000 nodes while NP-GL is 260 mW, DS-TPU is 1.6 W, and EADS is 1.2 W at the same reference scale. The 10^5× figure applies only against a 250 W A100 (nameplate TDP, not measured workload power), and DS-TS is actually worse than NP-GL and comparable to DS-TPU/EADS on the DSM side.
• No ablations. Three architectural innovations (EIC, SDN, HTI) are presented as the central contribution, but the paper contains no {EIC, SDN, HTI} on/off ablation, no sweep over SDN polynomial degree $D$ or HTI depth $L$, and no isolation of the E- and I-pathways from a single learned mixing. Because SDN's state-dependent polynomial can partly absorb what EIC's push–pull sign gating contributes, the case that all three mechanisms are individually necessary is not made empirically.
• Ground truth is forward Euler. Every benchmark in Appendix A.1 generates its reference trajectory with second-order centered differences and explicit forward Euler. Sub-$10^{-5}$ MAE against that reference does not imply physically accurate trajectories; it may indicate that DS-TS has learned to imitate the same Euler operator on the same grid, especially over 500 steps of a stiff or high-order system where Euler itself accumulates substantial error.
• Single-run "best" numbers. Table 1 bolds DS-TS in every one of 30 cells with no seeds, error bars, or confidence intervals. Some baseline gaps are within a small factor, and there is no way to know whether the ordering is robust. The reproduction below adds a specific reason to distrust that ordering: with a from-text reconstruction, DS-TS was strictly best in only 2 of the 14 comparable cells and NP-GL/EADS matched or beat it on Advection-Diffusion, Wave, and Hyperbolic.
• Empirical scope. All benchmarks are 1D or 2D on grids up to 1024 nodes total (32×32 in 2D). The introduction motivates DS-TS for time-critical applications that in practice are often 3D or much larger; the scope of the claims and the fully-coupled $O(N^2)$ resistor cost should be reconciled.
• Log-domain sign handling. The translinear log/exp pipeline is only defined for positive arguments, but the state-dependent coefficient $c_{m,i}(t)$ is a learned quantity that can clearly be negative, and the true per-branch sign is $\mathrm{sign}(c_{m,i}) \cdot \mathrm{sign}(I_{\text{IN}})^m$. As written, the coefficient-sign path is not described and per-degree parity is not addressed.
• EIC hardware vs. formulation. Section 3.4 says the E- and I-branches are obtained by sign-gating a single input current about zero, while Section 3.1.1 has $E$ and $I$ produced by independent learnable matrices $W^E$ and $W^I$. If the hardware description is literal, the two-matrix push–pull design collapses to a ReLU-like decomposition of one signal, undercutting the claimed added expressivity.
• Off-by-one in HTI order. The paper says an $L$-th order TDDE needs $L$ auxiliary temporal states, but the HTI chain $dx_i^{(\ell)}/dt = x_i^{(\ell+1)}$ with base $x_i^{(0)}$ and $L$ auxiliary states represents an $(L{+}1)$-th order equation. The convention is inconsistent with the standard reduction and with the paper's own chain, leaving the correct depth for e.g. the 4th-order Hyperbolic benchmark ambiguous (the reproduction had to pick $L=3$).
• Missing HTI chain equations. The reduction of the high-order TDDE to a first-order system is claimed but the chain equations themselves are referenced by prose and not displayed in the visible text; the mapping from the trained top-level operator to the target $F(\cdot)$ is left implicit.
• Rollout protocol. "Free rollout" is used for both DS-TS (continuous time, sampled) and AR-1/AR-16 baselines (discrete steps). The paper does not specify how the DS-TS trajectory is sampled to align with baseline outputs, whether any teacher-forcing is used, or how initialization is matched across methods.
• Mass conservation for Fokker–Planck. The Fokker–Planck benchmark requires $\int u\, dx\, dy = 1$; the paper normalizes the initial condition but does not describe how the constraint is preserved during a 500-step rollout, which is directly relevant to the reported LT MAE.
• QAT scope. Whether the 8-bit constraint applies only to the static conductances $W^B, W^E, W^I, W^C$ or also to the runtime state-dependent SDN coefficients and polynomial-expansion path currents is not stated; quantizing time-varying analog signals is materially harder than quantizing stored weights.
• Baselines punt on settings. "All baseline implementations use their originally published settings" leaves PINN loss weighting, FNO mode counts, UNet architecture, and other load-bearing choices per benchmark undocumented, and no numeric latencies/energies for the bar charts are tabulated.
• GPU power methodology. The 250 W A100 figure is a nameplate TDP, not a measured workload power, which inflates the energy ratio in the direction of DS-TS.
• Non-monotone resolution behavior on Hyperbolic. DS-TS short-term Hyperbolic MAE is 1.54e-6 (R1), 1.12e-6 (R2), 2.57e-6 (R3) — non-monotone in resolution and unexplained, and inconsistent with the monotone trend on the other four PDEs.
• Overloaded symbol. The letter $I$ is used both for the interior node set and for the inhibitory pathway current $I_i(t)$ within the same sentence.
• Related-work overreach. Framing prior DSMs as purely equilibrium-based is at odds with the paper's own DSM baselines (NP-GL, DS-TPU, EADS), which are characterized elsewhere as first-order dynamical systems that already exploit transients; the actual gap is high-order integration and dynamic spatial coupling.
• Relabelling of $\alpha$. $\alpha$ is defined once as the E–I signal strength and again in Appendix A.2 as "the residual gate scalar" without linking the two; a reader can transiently misread it as a separate mechanism.
• No classical numerical baseline. All baselines are ML surrogates or prior DSMs; a fine-timestep RK4/spectral/Crank–Nicolson reference on the accuracy–latency Pareto is missing, so the trade-off DS-TS is claimed to break has no visible classical benchmark.
• Training vs. rollout distribution shift. The training loss regresses $\hat S_\Theta(\mathbf{x})$ against $S^{\star}(\mathbf{x})$ at sampled states $\mathbf{x}^{(h)}$, but at inference the model is rolled out for 100 or 500 steps and encounters states off the training distribution. The sampling distribution is not stated and its effect on LT MAE is not analyzed.

Reproducibility & code

• Ordinal accuracy claim did not reproduce from the paper alone. A veritas execution reconstructed the method from the manuscript (no author code exists in the artifact bundle) and reproduced the qualitative ML-vs-DSM gap but not the paper's Table 1 ordering. Across the 14 (benchmark × resolution × horizon) cells where both DS-TS and all baselines completed, DS-TS was strictly best in only 2 (both Fokker–Planck R1); NP-GL and EADS matched or beat DS-TS on Advection-Diffusion, Wave, and Hyperbolic (geometric-mean ratios 1.02× and 1.08× relative to DS-TS), and FNO (AR-16) beat DS-TS by ~1000× on Reaction-Diffusion R3. Absolute DS-TS MAEs came out ~780× larger than the paper (O($10^{-3}$) vs O($10^{-6}$)), which is partly explained by the ~21 specification ambiguities the reconstruction had to guess and by a reduced training budget on this run — but the ordinal disagreement (parity with, rather than dominance over, the DSM baselines) is not explained by budget alone.
• Latency and energy headlines depend on an assumed DS-TS time unit. The manuscript never maps DS-TS's dimensionless simulation time to physical seconds, so a from-text reconstruction had to derive a 1 μs sample interval from the 1 MHz noise bandwidth in Sec. 4.4. With that assumption the reproduced ratios are ~8.1×10^3 for latency and ~1.8×10^6 for energy (about 8× and 18× above the paper); a ~15 μs interval would land energy near 10^5. Independently, the ML baselines here were timed on a contended A100 80 GB PCIe rather than the paper's A100 40 GB SXM, so the numerator is inflated too. The paper's headline speedup and energy-savings numbers cannot be re-derived from the manuscript alone.
• "Best baseline in every configuration" is contradicted by the reproduced numbers. The reconstruction directly tested noisy DS-TS vs the strongest baseline in each cell: DS-TS beat the best baseline in only 2 of the 14 cells where any baseline finished (both Fokker–Planck R1), and lost by three orders of magnitude on Reaction-Diffusion R3 LT. Noise is not the culprit — the noisy/noiseless degradation is only 1.001 on average, so noiseless DS-TS loses in the same cells.
• Noise pathway reproduces (favorably). The 52 dB current-domain SNR was recovered end-to-end (measured 52.000 dB over 1600 trials of the SDN-output injector), and the per-cell noisy/noiseless MAE ratios come out ~1.001 on average with a maximum of ~1.023 — well inside the paper's "less than 2×" statement.
• HTI mechanics reproduce; the comparative half of the "previously challenged existing DSMs" claim does not. The prescribed depths ($L=1$ for 2nd-order Wave, $L=3$ for 4th-order Hyperbolic) give finite, smoothly growing 500-step rollouts, and Wave short-term MAE lands within a factor of ~2.4 of the paper while Hyperbolic short-term MAE is actually an order of magnitude below the paper. But the prior-DSM baselines (NP-GL, DS-TPU, EADS) reach the same order of magnitude on both benchmarks, with EADS marginally beating DS-TS on all four Wave/Hyperbolic cells at R1, so the reproduction gives no evidence that these equations "previously challenged existing DSMs."
• Boundary conditions do hold across the rollout under the described tying. The reconstructed check (post-hoc re-tie disabled, so the diagnostic reflects the node dynamics alone) shows the seam-vs-interior curvature ratio decays over 500 steps in both a 1D order-1 and a 2D order-4 case, and never exceeds a genuinely periodic reference. The literal periodicity residual is uninformative on toroidal domains because the pairing is the identity, so this milder surrogate is what carries the claim.
• Absent artifacts: simulator, checkpoints, benchmarking scripts, Cadence testbench. No code-availability statement, no repository, no trained checkpoints, no benchmarking scripts, no Cadence netlists/testbenches for SDN, $G_m$-C, or the switch network. Every hardware-derived number (1.12 W power, 52 dB SNR, robustness plots, DS-TS latency) traces back to Cadence Mixed-Signal data that is not disclosed, and the prior-DSM baselines that Sec. 4.1 says were re-implemented "following the corresponding implementation settings" cannot actually be reconstructed because those settings are not in the manuscript.
• Missing hyperparameters. Neither the main text nor Algorithm 1 states the SDN polynomial degree $D$, the HTI depth $L$ per benchmark (compounded by the off-by-one convention), the training-set size $H$, the sampling distribution over states, the reference-solver $\Delta t$, or the DS-TS output sampling grid. Each of these is load-bearing for the reported MAE.
• R1/R2/R3 grid counts do not factor cleanly for the 2D benchmarks. Four of the five benchmarks are 2D on periodic squares, so 256/512/1024 total grid points map only ambiguously to 2D shapes (512 is not a perfect square). The 2D shape per benchmark is not stated.
• Figures 5–8 have no companion numerical table. The ~10^3× speedup, ~10^5× energy, and robustness bars are supported only by log-scale bar charts, without per-(method, benchmark) numbers or the A100 measurement protocol (batch size, precision, warm-up, NVML polling window).
• Node-scaling to 2000 nodes not documented. Power is reported "at a reference scale of 2000 nodes," but the actual benchmark $N$ is smaller and the scaling rule (linear? quadratic in the coupling matrix?) is not stated, which affects the DSM-to-DSM power comparison.
• Cadence noise model underdetermined. Eq. (16) needs per-block $H_i(f)$ and $S_{n,i}(f)$ that are not tabulated, the number of cascaded blocks $n$ per benchmark is not stated, and the reference signal amplitude used in the SNR numerator is not defined; the 52 dB is an input the paper hands to any reproducer rather than an output that can be recomputed.
• Periodic BC parameter-tying only in prose. The concrete parameter-tying rules (which entries of $W^B, W^E, W^I$ are shared, how 2D periodicity is enforced) are not written down.

Recommended Changes

Essential

• Scope the energy claim. Rewrite the abstract, contributions, and conclusion to state ~10^5× energy improvement against ML/GPU solvers specifically, and separately quantify the DSM-to-DSM comparison (where DS-TS is worse than NP-GL and comparable to DS-TPU/EADS at 2000 nodes). Addresses the "Energy overclaim" weakness.
• Add component ablations. Report DS-TS with each of {EIC, SDN, HTI} removed or replaced by a first-order baseline, and sweep the SDN degree $D$ and HTI depth $L$. Also isolate the E- and I-pathways from a single learned mixing to show they contribute beyond what SDN already provides. Addresses "No ablations."
• Report seeds and variance. Re-run every Table 1 cell over at least 3–5 seeds (for DS-TS and the ML baselines) and add mean, standard deviation, and whether the bolded winner survives a rank-based test. Addresses "Single-run best numbers" and the reproduced parity with NP-GL/EADS reported in the Reproducibility section.
• Use a higher-order reference or explicitly qualify MAE. Recompute MAE against RK4 (or spectral) at fine timestep for each benchmark, or state clearly that Table 1 measures error relative to a matched forward-Euler discretization on the same grid. Addresses the "Ground truth is forward Euler" weakness.
• Release code, simulator, checkpoints, and a Cadence testbench. Provide the software simulator, per-benchmark trained checkpoints, benchmarking scripts (including the A100 protocol used for Figures 5–6), and at least the top-level Cadence schematics/testbench used to produce the 1.12 W power and 52 dB SNR. Addresses "Absent artifacts" and "Cadence noise model underdetermined."
• Specify all missing hyperparameters and the DS-TS time unit. Add a per-benchmark table listing $D$, $L$, $N$, $H$, the reference-solver $\Delta t$, the DS-TS output sampling grid, and the sampling distribution over training states, together with the concrete PINN/FNO/UNet configurations used per PDE. Explicitly state the map from DS-TS's dimensionless simulation time to physical seconds so latency and energy ratios can be recomputed. Addresses "Missing hyperparameters", "Baselines punt on settings", and "Latency and energy headlines depend on an assumed DS-TS time unit."
• Fix the log-domain sign path and $W^E$/$W^I$ hardware description. In Section 3.4, specify the sign-recovery circuit for $c_{m,i}$, describe how per-degree parity is preserved in the polynomial output, and reconcile whether $W^E$ and $W^I$ are truly independent conductance networks or whether the sign-gating replaces the two-matrix formulation. Addresses "Log-domain sign handling" and "EIC hardware vs. formulation."

Suggested

• Fix the HTI order convention and display the chain equations. State the correspondence between the number of auxiliary states $L$ and the order of the target TDDE unambiguously, confirm the configuration used for the 4th-order Hyperbolic benchmark, and render the chain relations $dx_i^{(\ell-1)}/dt = x_i^{(\ell)}$ and the mapping between the trained top-level operator and $F(\cdot)$. Addresses "Off-by-one in HTI order" and "Missing HTI chain equations."
• Add a classical-numerical baseline. Include RK4 or Crank–Nicolson at fine timestep on the accuracy–latency Pareto so the trade-off DS-TS is claimed to break has a visible reference point. Addresses "No classical numerical baseline."
• Extend scope or scope the claims. Add at least one 3D or larger-grid benchmark, or explicitly restrict the abstract's framing to small-to-medium 1D/2D grids. Addresses "Empirical scope" and "Fully coupled correlation structure."
• Specify the rollout protocol. Define exactly how DS-TS's continuous-time output is sampled to match the discrete AR baselines, whether any teacher-forcing is used, and how initialization is aligned. Addresses "Rollout protocol."
• Describe mass conservation for Fokker–Planck. State whether $\sum u_{i,j}$ is renormalized every step, whether the loss includes a conservation penalty, and how much drift is tolerated over 500 steps. Addresses "Mass conservation for Fokker–Planck."
• Clarify QAT scope. State whether the 8-bit constraint applies to the SDN runtime coefficient currents and polynomial-path currents, not just the stored conductances. Addresses "QAT scope."
• Measure GPU workload power. Replace the A100 250 W TDP with an NVML-measured average during inference, and document the window. Addresses "GPU power methodology."
• Provide backing tables for Figures 5–8. Add per-(method, benchmark) numerical latencies, energies, and noisy vs. noiseless MAE. Addresses "Figures 5–8 have no companion numerical table."
• Explain the Hyperbolic non-monotone resolution behavior (or show it lies within seed variance). Addresses "Non-monotone resolution behavior on Hyperbolic."
• Document node-scaling to 2000 nodes and the 2D shape per resolution. State the actual $N$ used per benchmark, the rule used to scale to the 2000-node reference power, and the 2D grid shapes used at R1/R2/R3 for each 2D PDE. Addresses "Node-scaling to 2000 nodes" and "R1/R2/R3 grid counts do not factor cleanly."
• Retire the "purely equilibrium" framing. Rewrite the related-work paragraph so it distinguishes Ising-machine equilibrium readout from transient-exploiting DSMs (NP-GL, DS-TPU, EADS) and positions DS-TS's gap as high-order integration and dynamic spatial coupling. Addresses "Related-work overreach" and the reproduction finding that DS-TS ties rather than dominates prior DSMs on Wave/Hyperbolic.
• Disambiguate the overloaded $I$ and re-labelled $\alpha$. Use $\mathcal{V}_{\text{int}}$ for the interior node set and refer to $\alpha$ consistently as the E–I signal strength. Addresses "Overloaded symbol" and "Relabelling of $\alpha$."
• Write down the periodic BC parameter-tying rules explicitly. Addresses "Periodic BC parameter-tying only in prose."
• Discuss training-vs-rollout distribution shift. State the training-state sampling distribution and analyze how off-distribution states encountered during 500-step rollouts affect LT MAE. Addresses "Training vs. rollout distribution shift."