SAI
← All ICML 2026 orals

Lottery Prior: Randomized Neural Compression for Zero-Shot Inverse Problems

Haotian Wu, Di You, Pier Luigi Dragotti, Deniz Gunduz

OralReplication not startedPaper PDFOpenReview

Lottery Prior: Randomized Neural Compression for Zero-Shot Inverse Problems

SAI paper + code review · Referee report

Summary

The paper attacks zero-shot inverse problems (denoising, noisy super-resolution, inpainting) with a compression-based solver called Lottery Prior. Three ideas are coupled: (i) a randomly initialized coordinate-MLP whose weights are gated by a learned binary top-r_a mask (a DIP-style architectural prior in the lottery-ticket spirit of Wu et al., 2025); (ii) an entropy-constrained hierarchical latent modulation whose auto-regressive rate model imposes an implicit rate-distortion prior; and (iii) a lightweight ensemble across seeds and mask ratios at restoration. The conceptual move — treating a random subnetwork as a low-cost implicit codec and reading the trained system as an approximate compression-based ML estimator — is the most interesting idea and gives a natural information-theoretic story for the "less-is-more" behavior. The paper additionally derives non-asymptotic measurement- and signal-domain error bounds for the idealized compression-based ML estimator (fixed rate-R code, fixed distortion), with denoising a corollary that specializes cleanly against Zafari et al. (2026).

With a few kilo-parameters per instance, the method matches or beats strong zero-shot baselines (ZS-NCD, DIP, DD, ZS-N2N) on standard sets and extends cleanly to SR, inpainting, demosaicing, JPEG, and MRI — a real gain over AE-based patch-driven prior work. The main conceptual limitation is the gap between the guarantees and the deployed method: the theorems assume a fixed code over a signal class, whereas the estimator learns its codebook end-to-end from one degraded observation, so "provable guarantees" overstates what is on offer. Evaluation relies on single-point numbers for a stochastic ensemble with no per-seed variance, on ablations run on one or three images, on undefined "effective parameter" accounting, and on baselines taken as the max of reproduction and prior papers. Several cells are internally inconsistent (duplicate JPEG2K/DD entries, a BM3D collapse at Poisson α=25), and universal claims like "consistently best across all noise levels and datasets" are contradicted by the paper's own bolded entries. None individually threatens the core contribution, but together they thin the state-of-the-art case.

Strengths

  • Conceptual contribution. Casting a masked random network as a low-cost implicit codec and then reading the whole pipeline as an approximate compression-based ML estimator is a clean synthesis of Deep Image Prior, LotteryCodec, and rate-constrained denoising. It gives a plausible mechanistic explanation for why a few-kilo-parameter model can match much larger DIP/ZS-NCD backbones without needing early stopping.
  • Coverage beyond denoising. Prior compression-based zero-shot work is largely confined to denoising. Demonstrating the same objective works for noisy super-resolution, three flavors of inpainting, demosaicing, JPEG artifact removal, and MRI reconstruction is a substantive scope expansion.
  • Theoretical framing. The non-asymptotic bound in Theorem 3.1 with denoising (Cor. 3.3) and signal-domain (Cor. 3.5) specializations gives a two-term (distortion + rate-noise) picture consistent with the practical rate-distortion trade-off. The regime comparison with Zafari et al. (2026), while dense, is genuinely informative about when each bound is tighter.
  • Compute story. Table 6 shows a ~90× per-step MAC reduction over ZS-NCD, and Table 14 shows saturation by 5-10k steps. Even discounting the "effective parameters" label, the wall-clock and MAC/pixel picture is favorable and matches the paper's "good, fast, cheap" framing.
  • Component ablation is directionally clear. Table 8 shows the entropy model is load-bearing ("Not work" without it), the modulation network contributes ~2.4 dB, and the ensemble adds ~2 dB — enough to justify the design even if the exact numbers are noisy.

Weaknesses

  • Theory-practice gap on "provable guarantees." Theorem 3.1 / Cor. 3.5 assume a fixed code (f, g) with known rate R and worst-case distortion δ, and Cor. 3.5 additionally requires a restricted-injectivity constant α > 0 on the codebook difference set. The Lottery Prior codebook is learned end-to-end via a straight-through mask, a learned entropy model, and modulations, and α is never measured on the learned codebooks. The abstract/contributions language ("provable guarantees, including denoising as a special case") should be softened to reflect that the bounds motivate rather than certify the deployed system.
  • Overclaimed empirical superiority. Section 5.1's "consistently ... best across all noise levels and datasets" is directly contradicted by BM3D winning Set11 at σ=25 on both PSNR and SSIM and by ZS-NCD winning Set13 PSNR at Poisson α=15. Section 5.2's "highest PSNR across all test images" is contradicted by 8× "Bird" (BM3D+Bicubic 21.68 vs Ours 21.66). These are small margins but the wording is categorical.
  • No variance reporting for a stochastic ensemble method. The entire method is defined by randomness (initialization, top-k mask, ensemble across seeds and mask ratios), yet every number is a single point value. Gaps of 0.3-1.0 dB versus DIP, ZS-NCD, or RAM cannot be interpreted without seed-to-seed spread. This especially matters for the microscopy, MRI, and demosaicing tables (single-image or few-image comparisons).
  • Thin ablation footprint. Table 8's component ablation is on one image (Kodak01); Tables 9 and 10 use the first three Set11 images. Component-attribution claims (modulation network worth 2.4 dB; ARM width worth 0.26 dB) rest on very little data. Standard practice would be to rerun on a full test set with multiple seeds.
  • "Effective parameters" is undefined. The 0.7-3.3k count that anchors the "significantly fewer parameters" framing excludes, without saying so, the ARM (three 32-dim layers ≈ 2k params), the modulation network fθ (48- and 3-channel MLP), the score matrix P (same shape as W0), the multi-scale latent Z, and the Fourier reparameterization {Λ, B}. A like-for-like count against DIP/ZS-NCD would be substantially larger.
  • Baseline reporting rule is favorable. Baselines use "the better of our reproduction or the officially reported number," while the proposed method has no analogous best-of-two rule. Combined with unreported variance and small headline margins, this obscures whether the observed gaps are robust.
  • Theory-writing issues. In Appendix A.2 a stray "R_1" subscript contradicts the just-derived 2^{-ηR} probability; the one-sided tail-bound written for a general vector u is missing a ‖u‖² normalization; the factor-of-2 vs +2-in-exponent bookkeeping in Remark A.2 does not visibly add up to the claimed 2^{-η̃R+2} vs 2^{-ηR} shift; and the regime analysis asserts C_pre is minimized as R → ∞ without an explicit derivation. None of these are showstoppers, but they should be tightened.
  • Presentation and notation. "Mask ratio" is overloaded (subnetwork sparsity vs inpainting corruption ratio, both taking value 0.5), the random-network parameters flip between vector w_0 and matrix W_0, and the parameter set Ω is defined as {Z, ψ, θ, τ} but stored as {e, Ẑ, θ, τ} (dropping ψ, adding a seed). "J(τ) enforces a sparsity-based complexity constraint" is misleading given that r_a is a hyperparameter for top-k selection. Corollary 3.5's title flips between "general" (main text) and "linear" (appendix).
  • Table-level inconsistencies. Table 19 (Kodak, σ=15) has byte-identical JPEG2K entries for image 05 at σ=15 and σ=25, and byte-identical DD entries for images 15 and 16 at σ=15 — indicators of copy-paste error. Table 1's BM3D under Poisson α=25 collapses to 22.70 dB (worse than JPEG2K), strongly suggesting a mis-specified noise convention that should be documented, not left implicit. Table 12's "average of 0.85 dB on microscopy denoising" is actually the single σ=25 gap; the two-condition mean is 0.545 dB. Table 5's MS-SSIM averages both fall on the wrong side of the rounding boundary in the direction favoring Ours.
  • Ill-conditioned tasks are underclaimed but also under-diagnosed. The paper acknowledges that severely ill-posed operators (MRI, JPEG at Q=5) hurt gains, but frames this as an "information-theoretic" limitation of entropy regularization. The paper's own theory locates the difficulty in the injectivity constant α (identifiability), not in the rate term; this section should either invoke α or drop the information-theoretic framing.

Reproducibility & code

  • No code released at review time. The manuscript advertises only a project-page URL; no training/inference scripts, no ensemble seed lists, no saved (P, Ẑ, θ, τ) checkpoints, and no configuration files were available to inspect. Every claim's re-implementation therefore rests on the paper alone. This is a project-wide gap rather than a per-claim one, and it interacts with several missing implementation choices below.
  • Ensemble specification is incomplete. Section B.5 describes the aggregation but not the number, seeds, or step budgets of ensemble members per table. Table 6 mentions "5 mask ratios," and Section 5.5 says "5-10k steps," but the actual ensemble configurations used to produce Tables 1, 2, 12, and 13 are never pinned down.
  • Poisson-noise convention undocumented. BM3D's dramatic collapse at α=25 (Table 1) suggests the α-scaling convention (pixel range, transform) is non-standard or applied inconsistently across methods. Without an explicit statement in Section B.1, α-related comparisons cannot be re-run reliably.
  • Forward-operator implementations unspecified. The bicubic operator for SR (matlab-style vs torch, antialias flag), the Bayer CFA pattern for demosaicing (Table 4), the JPEG encoder/quality convention for AFHQ (Table 5), and the fastMRI undersampling mask (Cartesian, ACS lines, GRO, or variable-density — Table 13) are all left implicit. Since these change PSNR by 0.3-1+ dB, the reproducibility of the corresponding tables is compromised.
  • Split ambiguity for external datasets. "First image from each class" (AFHQ, Table 5) and "first 5 samples / randomly sampled 10" (Mouse Nuclei, Table 12) are not deterministic without split, ordering, and seed. Set5/13-based demosaicing test samples in Table 4 are described as "randomly selected" without seed.
  • Auxiliary metric implementations unspecified. LPIPS backbone (AlexNet vs VGG) and MS-SSIM implementation (skimage vs pytorch-msssim) are not stated, though they change values in Table 11 by 0.02-0.05.
  • "Effective parameter" counting rule is not defined, so the 0.7-3.3k anchor cannot be reproduced from the architecture description alone.

Recommended Changes

Essential

  • Soften "provable guarantees" language. Rewrite the abstract, the third contribution bullet, and the section-3 preamble to make explicit that Theorems 3.1 / Cors. 3.3, 3.5 apply to a fixed rate-R code, not to the learned Lottery Prior estimator. The learned-code connection is empirical and should be framed as motivation. (See Weaknesses: theory-practice gap.)
  • Report per-seed variance for the headline denoising, SR, and RAM comparisons. At minimum, mean ± std over three seeds for Table 1 (all six columns), Table 2 (Set5 averages), Table 4 (demosaicing), and Table 12 (microscopy). Where margins are within variance, downgrade the qualitative "state-of-the-art" wording. (See Weaknesses: no variance reporting.)
  • Release code, seeds, and configurations. Publish the training/inference implementation, the exact seeds and mask ratios used per table, the ensemble member counts and step budgets, and the checkpoints needed to regenerate at least Tables 1-3 and 6. (See Reproducibility: no code released.)
  • Correct the overclaimed universal statements. Change "consistently achieves the best performance across all noise levels and datasets" to reflect that BM3D and ZS-NCD win a handful of cells; change "highest PSNR across all test images" for SR to acknowledge 8× Bird. (See Weaknesses: overclaimed empirical superiority.)
  • Fix the Poisson-noise story. Document the α-scaling convention in Section B.1, state whether BM3D is applied with an Anscombe transform, and correct or re-explain the α=25 BM3D row of Table 1. (See Reproducibility: Poisson convention.)
  • Correct the Table 12 "0.85 dB" statement. Either replace it with 0.55 dB (two-condition mean) or restrict the sentence to σ=25. Similarly qualify "RAM improves substantially after fine-tuning" to reflect the σ=25 result. (See Weaknesses: table-level inconsistencies.)
  • Audit and correct duplicated table cells in Table 19 (JPEG2K image 05 at σ=15 and σ=25 are identical; DD images 15 and 16 at σ=15 are identical). (See Weaknesses: table-level inconsistencies.)
  • Define "effective parameters" and provide a like-for-like count that includes the ARM, modulation network, score matrix, and latent. Alternatively, retain 0.7-3.3k but report the full inference-stack count alongside. (See Weaknesses: effective parameters undefined.)
  • Report or bound the restricted-injectivity constant α on the learned codebooks for at least one super-resolution and one inpainting configuration, so Corollary 3.5 has an empirical anchor. If α cannot be characterized, present the corollary as motivational. (See Weaknesses: theory-practice gap.)

Suggested

  • Broaden the ablation footprint. Rerun Tables 8-10 on Kodak24 (or Set11) rather than one/three images, with per-seed variance. (See Weaknesses: thin ablation footprint.)
  • Clarify or replace "sparsity-based complexity constraint." Describe J(τ) explicitly as a fixed-cardinality (top-k) selection or as a genuine penalty term. (See Weaknesses: presentation and notation.)
  • Disambiguate "mask ratio." Introduce a separate symbol (e.g., ρ_obs) for the inpainting corruption ratio to distinguish it from the subnetwork mask ratio r_a. (See Weaknesses: presentation and notation.)
  • Unify notation. Pick either w_0 or W_0 for the random-network parameters and use it consistently; align the parameter set Ω = {Z, ψ, θ, τ} with the saved set in Algorithm 1. Align the title of Corollary 3.5 between main text and appendix. (See Weaknesses: presentation and notation.)
  • Tighten Appendix A.2. Remove the stray "R_1" subscript in the probability, restate the tail bound in unit-vector form (or add the ‖u‖² factor), spell out the factor-of-2 vs +2-in-exponent bookkeeping in Remark A.2, and show monotonicity of C_pre in R for the regime analysis. (See Weaknesses: theory-writing issues.)
  • Sanity-check PPT3 SSIM (Tables 17-18) and re-verify Table 5's MS-SSIM averages against higher-precision underlying values. (See Weaknesses: table-level inconsistencies.)
  • Symmetrize baseline reporting. Apply the "better of reproduction or reported" rule to both baselines and Lottery Prior, or drop it and use a single consistent rule. (See Weaknesses: baseline reporting rule.)
  • Document forward-operator and metric implementations (bicubic library and antialias, Bayer pattern, JPEG library, fastMRI mask type, LPIPS backbone, MS-SSIM implementation, Poisson scaling, AFHQ / Mouse Nuclei sample indexing). (See Reproducibility: forward-operator and metric implementations.)
  • Rewrite the ill-posed-task paragraph in Section 6 to invoke the injectivity constant α from Cor. 3.5 rather than a vague "information-theoretic perspective on entropy regularization." (See Weaknesses: ill-conditioned tasks.)