GoodDiffusion: Proactive Copyright Protection for Diffusion Generative Models via Learnable Sample-specific Signatures

Summary

The paper introduces GoodDiffusion, a proactive copyright-protection mechanism for diffusion bridge models (DBMs) that reverses the conventional backdoor-attack recipe: the protected model produces high-quality outputs only when the input carries a valid, owner-issued signature, and produces a fixed warning image otherwise. Framed against passive watermarking (only after-the-fact) and PCDiff-style degradation (still yields usable outputs), the conceptual move — internalizing authorization directly into the generative trajectory rather than gating it at the parameter or output layer — is the paper's main contribution and is genuinely novel for DBMs. A naive additive-signature realization is analyzed (Theorem 4.1) and shown to be white-box vulnerable via gradient-based surrogate recovery. To counter this, the authors introduce a Learnable Signature Network (LSN) — a UNet++ that maps each input to its own signature, blended with the input at strength $\gamma$ — with the intuition that sample specificity destroys the universality on which surrogate recovery relies. Experiments cover super-resolution, inpainting and deblurring on CelebA and ImageNet across four DBMs (DDBM-VP/VE, I2SB, DBIM); Abuse Rate is driven to $\approx 0$ while generation quality remains close to unprotected baselines, and a security experiment (Table 2, Fig. 4) shows that a static-signature surrogate is recoverable while a sample-specific one is not.

The direction is timely and the reverse-backdoor abstraction is a useful contribution. But the paper leans harder on Theorem 4.1 than the theorem can bear: Assumption A.2 (invertible linear corruption) does not hold for the actual super-resolution/inpainting/Gaussian-deblurring tasks, Assumption A.3 (Gaussian images) conflicts with the manifold argument the paper itself makes, and the security half of the story — that no efficient surrogate exists for LSN signatures — is only empirical against two narrow adaptive attacks, while Sec. B.6 shows the protection collapses after 50 fine-tuning steps and Sec. B.9 leaves 3-4% residual Abuse Rate on Stable Diffusion. The threat model also under-specifies whether the LSN weights $g_\phi$ are part of the leaked model, which materially changes the adaptive attack surface. Reproducibility is uneven: only the I2SB training path is actually released; DDBM/DBIM training, the surrogate attacks (Sec. 5.4/B.8), the fine-tuning driver, the AR/ER metric harness, and the T2I pipeline are all deferred; a compute-limited rerun of the I2SB code confirmed the CelebA deblurring row but produced 19% Abuse Rate on super-resolution rather than the paper's $< 0.06$%. The core idea deserves publication, but the theoretical and adversarial framing need tightening and the code coverage needs a substantial pass.

Strengths

• Conceptual contribution. Repurposing the backdoor mechanism from an attack primitive into a copyright-protection primitive is a clean and generative reframing; the "selectively permissive, otherwise closed" phrasing is a good way to articulate a use-time model-level guarantee that is not achievable by watermarking or PCDiff-style degradation.
• Sample-specific signatures are a principled fix to the surrogate-recovery vulnerability. The observation that a fixed signature amounts to a global manifold translation, while an LSN produces an input-dependent deformation, is a compelling argument for why the naive design is vulnerable and the LSN is qualitatively harder to attack universally.
• Coverage of tasks and bridges. The experimental sweep across three I2I tasks and four DBMs (DDBM-VP/VE, I2SB, DBIM) on two datasets is broader than most protection papers, and the appendix's fine-tuning attack, adaptive surrogate LSN, T2I extension, and $\gamma$ ablation together give the reader an unusually honest picture of where the method holds and where it breaks.
• Threat model articulated up front. Sec. 3.3 explicitly names the strong white-box thief and the "separation of duties" assumption for the signature service, which is more precise than most watermarking papers, even if (see Weaknesses) the precise trust root and the status of the LSN weights need to be sharpened.
• Executed code path reproduces the qualitative claim on I2SB deblurring. When run end-to-end, the shipped I2SB pipeline drives unauthorized CelebA-deblurring Abuse Rate to 0.0%, inside the paper's headline range — the core mechanism does work when trained enough.

Weaknesses

• Theorem 4.1 assumptions do not cover the actual tasks. Assumption A.2 requires an invertible linear corruption operator, but 4x pooling for super-resolution is rank-deficient, freeform inpainting has a null space, and Gaussian deblurring is ill-conditioned. Invertibility of $\mathbf{A}$ is used at the last step of the proof (Eq. 21) to conclude $\mathbf{N}^{*} \approx \mathbf{k}$, so under the paper's own experimental settings the theorem's conclusion is not directly justified.
• Theorem 4.1 has internal tension. The theorem asserts the score functions "perfectly match" but the recovered $\mathbf{N}^{*}$ only "approximates" $\mathbf{k}$; under the stated assumptions the conclusion should be equality, and the passage from equal scores to equal perturbations needs an injectivity/identifiability step that is not stated.
• Theorem is set up for additive injection, but the deployment is affine. The theorem's thief perturbs $\mathbf{x}_1$ additively, while the deployed sample-specific injection is $(1-\gamma) g_\phi(\mathbf{x}_1) + \gamma \mathbf{x}_1$; a knowledgeable thief would parameterize their surrogate accordingly (surrogate LSN + blend at $\gamma$), and the paper never analyses this natural attack.
• Asymmetric evidence for the paper's central claim. Static fragility is a theorem; LSN robustness is only argued empirically against (i) a global surrogate perturbation and (ii) a naively-trained surrogate LSN. Neither covers a per-input inversion attack — for each query $\mathbf{x}_1$, optimize a signature for that sample only — which is the natural analogue of Sec. 5.4's attack in the sample-specific regime and precisely what trigger-inversion literature does.
• Fine-tuning defeats the protection in ~50 steps. Sec. B.6 shows AR = 100% after 50 fine-tune iterations at batch 256, i.e., a few GPU-minutes for a thief with weights and a small labelled set. The framing that "infringers may not have the resources" is unpersuasive; this result is arguably more consequential than the main-text security narrative and belongs in Sec. 5.
• T2I extension shows the guarantee is not "complete". Table 8 reports AR = 3.3-4.5% for SD V1.4/V1.5 — roughly 1 in 25 unauthorized queries still yields a usable image. This directly contradicts the abstract's "refuses to generate" and the introduction's "completely prevents" framing.
• Threat-model overclaim and ambiguity. The paper emphasises a "strong white-box" adversary but the security rests on the signature service being external — what is protected is the signing key/service, not the model per se. The paper is also silent on whether the LSN weights $g_\phi$ ship with the model to the authorized user (they must, for inference) and, if so, whether the thief in Sec. 5.4/B.8 has $g_\phi$.
• No comparison against PCDiff or watermarking baselines. PCDiff is repeatedly invoked as the reason for cutting off rather than degrading unauthorized generation, but Tables 1/3/9 report only unprotected vs. GoodDiffusion. Without at least PCDiff and a strong watermarking baseline in the same tables, the operational value of "warning image" behaviour over PCDiff-style degradation is not demonstrated.
• AR/ER threshold undefined. AR and ER are defined by an MSE-to-target/warning threshold, but the numerical threshold is never specified in the paper or the code. Since every headline number is threshold-dependent, this leaves the metric unauditable.
• Only $\gamma$ is ablated, and on a single task/bridge. $\pi_k$, the warning-image choice, and the LSN backbone are load-bearing and never varied. The warning-image ablation is particularly important because the entire unauthorized branch is optimized against one fixed target.
• No seed variance. Every table appears to be a single training run per configuration; several differences (e.g., ER = 0 vs. 0.06 vs. 0.31, SSIM 86.18 vs. 86.80, D-Surr. AR = 0 vs. 0.04) are small enough that a reader cannot tell whether they are real.
• Notation drift and surface issues. Symbol $\bar{\mathbf{x}}_0$ has opposite meanings between Sec. 3.2 (attacker target) and Sec. 4.2 (defender warning); $\pi$ becomes $\pi_k$ without redefinition and is described as "authorized vs. unauthorized" in the appendix but "authorized vs. warning" in the main text; Assumptions A.2/A.3 swap the input/target labels; Table 6 lists "DDIM" instead of "DBIM"; the running header says "Diffusion Generative Models" while the title says "Diffusion Bridge Models".
• Recursive metric names and undefined reference distribution. Table 2 introduces "bad BPSNR (BPSNR)" and "bad BSSIM (BSSIM)" (recursive), and does not state the reference distribution/image for BFID/BPSNR/BSSIM — critical because FID's direction reverses with the reference.
• Learning-theory citation does not license the objective. Mao et al. (2023) concerns classification surrogate-loss consistency; it says nothing about the mixed authorized/warning score-matching objective in Eq. 7.

Reproducibility & code

Veritas ran the released repository (https://github.com/qsx830/GoodDiffusion) in full-execution mode. Only one of fourteen tracked claims produced a verdict; the remaining thirteen could not be attempted because the required code paths, attack scripts, or artefacts are not shipped.

• Headline aggregates. The paper's claim that GoodDiffusion holds Abuse Rate below 0.06% on CelebA super-resolution and deblurring across all four bridges could only be partially checked. On the sole reproducible pipeline (I2SB), CelebA deblurring reproduced (AR = 0.0%, inside [0, 0.06]) but CelebA super-resolution came in at AR = 19.0%. The reproducer's training budget was ~100-150 iterations rather than the paper's implied ~10^6, so this is best read as evidence that the headline Abuse Rate is sensitive to training compute in a way the paper does not surface, not as evidence that the method itself fails.
• Coverage of bridge models. DDBM-VP, DDBM-VE, and DBIM training/generation are marked as TODO in the released code. That means 6 of 8 protected AR values in Table 1, all of Table 3 except the I2SB row, and 8 of 12 rows in Table 9 cannot be reproduced from the shipped repository.
• Security experiments. The Eq. 5 surrogate-recovery attack (Sec. 5.4, Table 2 S-Surr./D-Surr. rows, Figs. 4/11/12) has no shipped CLI entry point, and the static-signature training path itself is absent — only the dynamic/LSN branch ships. The adaptive surrogate-LSN attack of Sec. B.8 / Table 7 similarly has no attacker training loop in the repository. The fine-tuning attack of Sec. B.6 / Table 5 exposes only a _backward_single_finetune_attack_loss helper without a driver, so the crossover at step 50 cannot be independently verified.
• Warning image. The training path loads the warning image from a hard-coded absolute path under the first author's home directory. Because the entire unauthorized branch is optimized against that specific image, any substitute changes every AR/ER/FID/PSNR/SSIM number.
• Metric harness. No script in the repository computes AR or ER from generated images; the MSE threshold that defines both metrics is not stated in the paper or the code. During the veritas reproduction it had to be re-implemented (per-image MSE < 0.01 to clean GT), which is not necessarily what the paper used.
• T2I extension. The Stable Diffusion V1.4/V1.5 pipeline in Sec. B.9 / Table 8 (prompt-signature loader, CLIP-Score, MS-COCO reference statistics) is entirely absent from the released repository.
• Checkpoints and datasets. No pretrained checkpoints of GoodDiffusion or any of the four unprotected baselines are shipped, and datasets must be staged locally. The only thing a reader can verify without substantial re-implementation is that a UNet++ LSN plus I2SB compiles and runs.

Recommended Changes

Essential

• Restrict or generalize Theorem 4.1 to the actual tasks. State it for invertible $\mathbf{A}$ and discuss what happens under the non-invertible super-resolution/inpainting operators actually used, or extend the proof (e.g., pseudoinverse arguments) to cover them. Along the way, add the identifiability step from equal scores to equal perturbations and align the "perfectly matches / approximates" qualifier with the assumptions the proof uses.
• Extend the theorem's threat model to the deployed blended injection. State whether the security analysis covers $(1-\gamma) g_\phi(\mathbf{x}_1) + \gamma \mathbf{x}_1$ or only pure additive $\mathbf{N}$; if the latter, extend it.
• Evaluate a per-input inversion attack on the LSN. For each query, optimize a per-sample surrogate signature against the frozen $s_\theta$ (analogue of Sec. 5.4 but not universal); report AR/BFID and compare to the S-Surr./D-Surr. entries in Table 2. This closes the asymmetry between the theorem (fragility) and the merely empirical robustness argument.
• Promote the fine-tuning result to the main paper. Move Sec. B.6 into Sec. 5, quantify attacker cost in wall-clock and dollars, and either propose a mitigation or explicitly scope the guarantee to "no fine-tune access".
• Add PCDiff and a watermarking baseline to Tables 1, 3, 9. Without a head-to-head comparison against the paper's stated competitor, the operational value of cutting-off vs. degradation cannot be judged.
• Define the MSE thresholds for AR and ER numerically and include a sensitivity sweep, or replace the threshold-based metric with a threshold-free one.
• Soften the "completely prevents / refuses to generate" language to a resource-bounded statement, and surface the T2I 3-4% Abuse Rate in the abstract/intro rather than only in Sec. B.9.
• Clarify whether the LSN weights $g_\phi$ are part of the leaked model and, if so, revisit the Sec. 5.4/B.8 attacks under that assumption. Reframe "strong white-box" as "white-box on the model, black-box on the signature service".
• Release the DDBM-VP, DDBM-VE, and DBIM training/generation code (or explicitly scope the headline claims of Tables 1/3/9 and Fig. 3 to I2SB). Release the static training path, the surrogate-signature attack script (both static and sample-specific), the adaptive surrogate-LSN training loop for Sec. B.8, the fine-tuning driver for Sec. B.6, and the AR/ER/FID/PSNR/SSIM harness with the MSE threshold hard-coded.
• Ship the warning image used to train the unauthorized branch (or a byte-reproducible generator), and replace the hard-coded absolute path with a config-driven one.

Suggested

• Add an AR-vs-training-iterations curve for I2SB so readers can see how quickly the protection consolidates — this is the axis on which the compute-limited reproduction of the super-resolution row broke.
• Report per-seed variance (mean $\pm$ std over at least 3 seeds) for the CelebA I2SB rows in Tables 1, 3, and 4, so small ER/SSIM differences (and the Table 2 D-Surr. deblurring 0.04 entry) can be interpreted.
• Ablate $\pi_k$, the LSN backbone, and the warning-image content. The warning-image ablation is particularly important because the whole unauthorized branch is optimized against a single fixed image.
• Provide at least one end-to-end pretrained checkpoint for GoodDiffusion + I2SB with the exact preprocessing pipeline, so an external reader can verify the reported numbers without a full retrain.
• Release or explicitly deprecate the T2I code. If Sec. B.9 stays in the paper, ship the SD V1.4/V1.5 integration; otherwise move Table 8 to "preliminary" and drop it from the main-text protection claims.
• Fix the surface issues. Align the running header with the title; use a distinct symbol for the warning image (e.g., $\mathbf{w}$); reconcile $\pi$ vs. $\pi_k$; correct Assumptions A.2/A.3 input/target labels; correct "DDIM" to "DBIM" in Table 6; clarify "bad BPSNR (BPSNR)" and state the reference distribution for BFID/BPSNR/BSSIM.
• Downgrade the "if and only if" claim for (G1) to a target property that Sec. 4.5 argues empirically, since Eq. 7 only enforces the "if" direction.
• Remove or hedge the "SSIM slightly surpasses the normal baseline" remark unless supported by multi-seed error bars.
• Trim the learning-theory citation. Mao et al. (2023) does not support the mixed score-matching objective; either drop it or replace it with a directly-relevant reference.