On the Identifiability of Poisson Branching Structural Causal Model Under Latent Confounding
SAI replication review · Referee report
Summary
This paper extends the Poisson Branching SCM (PB-SCM) framework to admit unobserved common causes, calling the result LC-PB-SCM. The conceptual move is to translate branching-structure identifiability under latent confounding — a problem where classical linear-non-Gaussian and rank-based tools do not apply — into an algebraic question on the probability generating function (PGF) of count data. The authors show that every monomial in the log-PGF is in one-to-one correspondence with a Pruned Trie obtained by pruning a rooted prefix tree over directed paths of the causal graph. Because the coefficients of these Trie-generated monomials are products of edge-thinning probabilities that survive marginalisation of exogenous latent roots, the monomial set becomes a structural fingerprint one can read off empirical partial derivatives of . Local-PGF restrictions collapse the analysis to three observed variables at a time, yielding orientation rules (Theorems 3.7–3.12) that the authors argue are complete under monomial equivalence for the 29 canonical 3-node classes. A three-phase algorithm (skeleton, confounded-edge, triple orientation) is derived at cubic complexity and evaluated on synthetic and two real count-data settings. The contribution is genuinely novel: it is the first identifiability treatment of PB-SCMs with latent confounders, and the Trie/monomial correspondence is an elegant, non-obvious bridge between combinatorial structure and PGF algebra. Executing the released code confirms this theoretical core reproduces cleanly — the symbolic log-PGF, the empirical estimators, the discriminating-monomial argument, and the Case-2 structure recovery all match the paper. The main conceptual limitation is that the entire result is scoped to exogenous, mutually-independent latent roots and to local 3-variable substructures; latent mediators, hierarchies, and any identification needing -variable PGF terms are deferred. Several proof steps (Types (d1)/(d3), completeness Types 16/17/18) do not close as written, and — a new finding from execution — the headline synthetic F1 numbers did not reproduce and the comparative "beats all baselines" claim is untestable from the artifact.
Strengths
- Conceptual contribution. The Trie / log-PGF-monomial correspondence (Theorem 3.5) is a clean and, to my knowledge, new bridge from branching-process structure to a symbolic-algebraic fingerprint that survives marginalisation of exogenous latent roots. The "each Pruned Trie is one term in the log-PGF" intuition is genuinely useful, and executing the shipped symbolic code reproduces the running-example log-PGF byte-for-byte.
- Identifiability of confounded direct edges. The result that a bow (a direct edge coexisting with a latent common cause of its endpoints) becomes identifiable via the signature is non-trivial: this configuration is generically non-identifiable in linear-Gaussian and rank-based settings, and the paper turns it into a testable third-derivative condition. The coefficient estimator for this signature converged to its true value in replication.
- Complete case analysis for local 3-node structures. Enumerating the 29 monomial equivalence classes and pairing each to an orientation rule is exactly the granularity a reader wants, and it makes Phase 3 a mechanical lookup once the empirical monomial set is estimated.
- Practical polynomial-time algorithm. The complexity was confirmed by code inspection of the shipped loop structure (pairwise, edge-wise, and triple enumeration), making the method plausibly usable at moderate .
- Reproducible empirical-PGF backbone. The empirical PGF and its third-derivative estimate concentrate with the reported decreasing-variance trend; running the shipped estimators reproduced Tables 6 and 7 in trend and magnitude, giving a reassuring numerical baseline for the Phase 1–3 tests.
Weaknesses
- Type (d1) proof is internally inconsistent with its conclusion. The proof explicitly admits an alternative where the third comes from an additional latent edge with non-adjacent — a case with no – edge — yet concludes , which presupposes an adjacent arrowhead. Since Theorem 3.13 (completeness) rests on each case closing, this must be repaired or the conclusion softened.
- Type (d3) does not exclude a direct latent edge to . The argument that " appearing once" forces (hence ) does not rule out a direct with no edge; both produce a single copy.
- Completeness entries for Types 16/17/18 look inconsistent. Type 16 derives from the presence of while Type 17 derives the same from its absence; Type 18 claims a directed from just the adjacency monomial , whereas everywhere else a confounded orientation requires the higher-order (Theorem 3.8). These read as citation errors or missing intermediate monomials.
- Theorem 3.9 is missing its monomial condition. The body announces "under the following monomial condition:" and gives none; the conditions live only in Figure 3 subcaptions, inconsistent with Theorems 3.10–3.12.
- Types (e1)/(e2) collapse to the same condition. Both invoke but conclude opposite-strength targets; the distinguishing structural context exists only as figure drawings. Types (e3)–(e5) inherit similar under-specification.
- Two proof cruxes are appeals to intuition. Theorem 3.8's necessity (" would essentially be disconnected") and Type (c1)'s "" sub-case ("must pass through since are adjacent") do not formally exclude latent-only alternative routes; and Type (c1) cites Theorem 3.7 to orient an edge, which that skeleton theorem cannot do.
- Restriction to exogenous latent roots is a substantive scope limit. Latent mediators, latent-to-latent chains, and hierarchies are excluded by Definition 2.1 — precisely the settings much compared work targets. The framing ("bridge this gap") suggests broader coverage than the assumptions deliver.
- Unspecified test protocol and no misspecification analysis. The bootstrap/threshold significance level and any multiple-comparison control across the tests are never stated; absence-based rules (e3, e5, f1–f4) lean silently on faithfulness at exactly the finite-sample regime where they are most fragile. Appendix E studies only convergence under a correctly specified model, with no stress test under over-dispersion, zero-inflation, or additive contamination.
- Evaluation protocol changes between tables. The random-graph experiments switch to a laxer F1 (circle marks compatible) scored against the PADMG rather than the PPADMG, and to a much sparser generative regime (), so F1 values are not comparable across tables and the change is not motivated.
- Real-world experiments are minimal. The shopping-mall setting reduces to a single two-variable orientation once Impressions is declared latent (none of the local 3-node rules run), and the football setting is handed the true skeleton, so Phase 1 is never exercised on real data.
- Global-reconstruction claim overstated. The paper concedes global identifiability needs extra information, then claims local enumeration can "reconstruct the overall causal graph"; overlapping-triple conflicts and -variable edges are unaddressed, and Phase 2's asymmetric third derivative is applied as if symmetric.
- Minor consistency issues. Latent-mediator equivalence in the main text omits the mediator's Poisson-noise absorption that Appendix F correctly performs;
LC-PBSCM(abstract) vsLC-PB-SCM;Des(i)written as membership in ; bold reused for both latent confounders and the local-PGF target set; Definition 2.5's domain excludes the latent indices the proofs use.
Reproducibility & code
- Theoretical core reproduces exactly. Running the shipped
PGF.py,util.py, andpgf_confounder_partial.pyunmodified reproduced: the running-example log-PGF to the last digit (); the empirical PGF converging to the true with the reported decreasing variance (Table 6); the coefficient converging to (Table 7); the motivating-example monomial-set difference being exactly ; and the Case-2 recovery , . The inference engine is present, runnable, and correct. - Headline Table 1 numbers did not reproduce. Under the paper's stated protocol the reproduced "Ours" F1 was versus the reported — systematically 0.11–0.27 lower, with only Case 2 within one reported . Qualitatively the correct PPADMG is recovered on representative repeats, so the gap most plausibly reflects the from-scratch, strict endpoint-mark F1 the reproducer had to write (the paper's own metric, per Appendix C.1, is laxer) and figure-transcribed case topologies — but the exact numbers stand unverified.
- The comparative claim is untestable from the artifact. None of the four baselines (FCI, PB-SCM, GES, OCD), the F1 routine, or the ground-truth-PPADMG builder ship, so the "outperforms all baselines" half of the headline result and all of Tables 2–4 (additional cases and 10-/20-variable sweeps) could not be attempted. Reproducing any comparative cell requires re-implementing four external baselines and the full evaluation harness.
- Real-world results could not be attempted. Both datasets require authenticated download and no loader/preprocessing script maps raw rows to the count matrices used (Clicks/Conversions binning; per-match Y1/Y2/R/S aggregation; latent handling of Impressions/Foul). An independent reproduction would start from different counts.
- The 29-class enumeration and Figure 5 are not scriptable.
graph2pgf/pgf2termscompute per DAG (and matched representative classes exactly), but no enumeration/isomorphism-canonicalisation script emits the 29 classes, and no figure-generation code exists; the per-type monomial-set displays are also missing from the manuscript, so Theorem 3.13's case-by-case content cannot be checked as printed. - Robustness tables have primitives but no driver. The utilities behind Tables 6–7 are shipped and reproduced, but no driver fixes seeds or runs the sweeps, and Table 7's evaluation point is not stated in the code path.
Recommended Changes
Essential
- Fix the Type (d1) proof. Either rule out the "additional with non-adjacent" case by a monomial argument on the other terms it would introduce, or weaken the concluded orientation.
- Fix Type (d3) step 1. Add a monomial argument that positively excludes a direct latent edge under the (d3) context, so " appearing once" genuinely implies .
- Reconcile completeness Types 16/17/18. Exhibit the differing surrounding monomials that let presence (Type 16) and absence (Type 17) of both yield , and correct the Type 18 citation (likely Theorem 3.8 via ) or supply the missing higher-order monomial.
- Restore the missing content. State Theorem 3.9's monomial conditions inline; restore the per-type monomial-set displays in the Type 1–29 completeness proof; and state the structural context of Types (e1)–(e5) inline so each rule is self-contained.
- Release the evaluation harness and disclose what is missing. Ship the four baselines (or wrappers plus commit hashes), the exact F1 metric, the ground-truth-PPADMG builder, the Case 1–9 graph specifications, and the random-graph drivers for the 10-/20-variable sweeps, so the headline Table 1 numbers and the comparative Tables 2–4 can be checked. Add an artifact note stating the baseline/metric/data layers are not otherwise included.
- Reconcile the reproduced Table 1 shortfall. Report results under the paper's own (documented) metric and confirm the cells regenerate; if the strict-vs-lax metric explains the gap, define the metric precisely and report both.
Suggested
- Add a misspecification stress test. Rerun the synthetic tables with negative-binomial noise or additive contamination to show the presence/absence monomial tests degrade gracefully — addressing the missing-robustness weakness.
- Specify the test protocol. State the bootstrap/threshold significance level, the evaluation point , and how error is controlled across the simultaneous tests; note faithfulness reliance where absence-based rules (e3, e5, f1–f4) are stated.
- Provide real-world preprocessing scripts and strengthen the real experiments. Map the raw Kaggle/football files to the count matrices used, and add a real dataset with more observed variables so Phase 1 and the local 3-node rules are actually exercised (currently the shopping-mall case is a two-variable orientation and the football case is given the skeleton).
- Hold the evaluation metric and generative regime fixed, or label them. Justify the / PADMG / compatible-circle changes in the random-graph experiments so cross-table comparison is valid.
- Document Phase 2's pair ordering. State whether both orderings of the asymmetric third derivative are tested and how conflicts are broken, and mirror this in the code; rename the phase to reflect that it detects confounded edges (bows), not all latent confounding.
- Soften the global-reconstruction claim to "build up a local estimate", and describe how overlapping-triple conflicts are resolved.
- Complete Theorem 3.8's sufficiency with the coefficient-non-vanishing/faithfulness invocation used in Theorem 3.7, and make the two intuition-based cruxes (Theorem 3.8 necessity; Type (c1) ) into formal arguments; fix the Type (c1) cross-reference (Theorem 3.7 3.8).
- Fix minor consistency issues. Harmonise
LC-PBSCM/PBSCM; rewriteDes(i)as a directed-path condition; rename the local-PGF target set (e.g., ) and extend Definition 2.5's domain to ; state the main-text latent-mediator equivalence "after absorbing 's Poisson noise".