Source-linked AI summary

Operationalising Weaker Faithfulness: Recovery, Ambiguity and Computational Limits

Cillian Hourican, Rick Quax

arXiv:2610.31724v1stat.MEcs.LGstat.ML

TL;DR

Jointly informative relationships can be missed when structure learners test variables separately. The paper studies coordinated candidate protection and explicit search over alternative DAGs; it reports conditional recovery gains, with extra errors and substantial limits from finite evidence and computation.

  • Problem

    Jointly predictive variables may each be individually independent of an outcome, allowing marginal-association tests to discard informative links.

    Source: §1 · p. 1
  • Method

    The paper combines selective protection in an adjacency learner with a separate estimator that retains qualifying alternative DAG explanations.

    Source: §1 · p. 2, §2.2 · p. 3
  • Results

    The methods achieve conditional gains: coordinated changes recover joint links, and direct family search sometimes preserves explanations excluded by a selected skeleton.

    Source: §1 · p. 2, §5 · p. 12
  • Takeaways & Limitations

    Coordinating candidate survival and later protection can recover jointly informative links, while family search can preserve alternatives that a selected skeleton loses.

    Source: §4.2 · p. 11, §5 · p. 12

    The gains coexist with extra edges, false exclusions, finite-sample error, and computational failures, leaving reliable inference at larger scales unresolved.

    Source: §2.2 · p. 4, §4.1 · p. 11, §5 · p. 12

Abstract

from arXiv · show

Variables can predict an outcome jointly while each is individually independent of it, causing structure learners to discard informative links. We investigate how to operationalise weaker faithfulness through two approaches: selective protection in an adjacency learner and explicit retention of alternative directed graph explanations. Matched experiments show that changes at both an early candidate-removal stage and later pruning can be necessary. In noisy XOR with a descendant (N = 10,000, 20 matched samples), combining reactivation and protection raises complete parent-pair recovery from 0/20 for either change alone to 20/20, while adding false edges. A confirmation study shows that the family estimator can preserve explanations lost by orienting the protected learner's selected skeleton: in one weaker signal XOR condition (N = 10,000) it retains the entire reference family in 48/50 samples, compared with 0/50 for the skeleton adapter. These gains are conditional. Protection can add only error, weaker effects produce false exclusions, and an ordinary faithful-chain comparison favours the PC learner. Complete nonempty family outputs decline at six/eight variables, and all 400 twelve/twenty-variable searches remain unfinished at the reporting cutoff. Neural controls show engineered retention, while the family estimator returns no accepted graph within its application budget. This operationalisation identifies two deletion stages that must be coordinated in the motivating example and shows when direct family search preserves explanations lost by selecting one skeleton. Finite evidence and computation remain substantial barriers to reliable general inference.

1 Introduction

Noisy XOR shows how jointly informative relationships can disappear when learners test variables only through marginal associations. The paper investigates coordinated changes to candidate deletion and direct family search, finding conditional recovery gains alongside finite-sample and computational limits.

1 Introduction

  • In noisy XOR, each parent is individually independent of the outcome although the pair predicts it, so marginal-only deletion can erase the relationship.The example has strictly positive probabilities and violates faithfulness to its generating collider.Source: §1 · p. 1
  • Operationalising weaker faithfulness requires choices about candidate search, deletion, finite evidence, and what graph an algorithm returns.Weaker-faithfulness assumptions allow some relationships to be supported by triple interactions rather than persistent pairwise dependence.Source: §1 · pp. 1–2
  • The paper compares a protected adjacency learner that selects one skeleton with a family estimator that retains alternative directed graphs, including different skeletons.The adjacency approach reconsiders early removals when further conditioning restores dependence and selectively protects surviving links from later pruning.Source: §1 · p. 2
  • The experiments combine matched stage comparisons, exact small-law references, a confirmation study, and an application on a fixed neural background.They ask which deletion stages must change, what explicit alternatives preserve or lose, and how gains hold under finite samples and larger problems.Source: §1 · p. 2
  • The results show conditional gains: coordinated deletion-stage changes recover joint links, while family search can retain alternatives lost by selecting one skeleton.The benefits coexist with extra edges, false exclusions under weak evidence, and severe limits on larger searches; neural experiments test operational reach, not biological causal truth.Source: §1 · p. 2

2 Methods

The methods coordinate changes to candidate removal and pruning, and separately define a search that retains alternative DAG explanations. Experiments compare these approaches across controlled designs while distinguishing reference-family recovery, practical search completion, and causal interpretation.

2.1 Retaining joint relationships through an adjacency search

  • Grow admits candidates, shrink tests their removal, and global pruning tests assembled edges; reactivation can veto a shrink removal when added conditioning restores dependence.Reactivation revisits a proposed removal; it does not recover candidates already discarded.Source: §2.1 · p. 2
  • Protection uses a joint-information contrast and a fixed 0.3-bit threshold to shield existing edges from pruning, but cannot add absent edges.The threshold is an effect-size choice, not a significance level or causal certificate; the factorial comparisons isolate stage changes.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 From a selected skeleton to alternative graph explanations

  • The target family retains all DAGs satisfying Markov compatibility, the weaker adjacency condition, subgraph minimality, and supplied restrictions, rather than selecting a sparsest graph.Small known laws are searched by exhaustive DAG filtering and an ordering-based construction that supplies subgraph-minimal DAGs.Source: §2.2 · p. 3
  • Orienting one selected skeleton cannot recover all three noisy-XOR explanations, while causal interpretation additionally requires assumptions not established by enumeration.The practical family estimator uses finite CI decisions; completion does not imply recovery of the exact-law family.Source: §2.2 · pp. 3–4, Figure 1 · p. 4

2.3 Experiments and comparison targets

  • The experiments use matched stage comparisons, a confirmation study, exact small-law references, and comparisons with PC-stable in independently classified faithful conditions.Larger experiments lack full exact population-family references, and above-four-variable scaling laws lack verified classification under the chosen assumptions.Source: §2.3 · p. 5

2.4 Time order and the neural application

  • Time-order experiments distinguish relabelled static columns from independent episodes and overlapping trajectories; the neural application uses sampled transitions without known causal ground truth.The family application combines six neural units at both slices with synthetic modules; its full 36-column panel is unsupported.Source: §2.4 · p. 5

2.5 Outcomes, uncertainty and reporting boundaries

  • Outcomes distinguish whole-family coverage from planted-graph membership and condition false-assertion rates on complete nonempty outputs.Unfinished searches remain in planned-input accounting, and intervals are descriptive rather than simultaneous superiority tests.Source: §2.5 · p. 6

3 Results

Coordinating reactivation and protection recovers noisy-XOR parent pairs that either change alone misses, but adds false edges. Family search can preserve explanations lost by selecting one skeleton, although gains are conditional and limited by finite evidence and computation.

3.1 Coordinating deletion stages recovers relationships

  • 20/20 combined reactivation-and-protection runs recovered the complete parent pair, versus 0/20 for each single change.In the same samples, protection lowered mean skeleton precision from .975 to .738 and added 1.05 false adjacencies on average.Source: §3.1 · p. 6, Figure 2 · p. 7

3.2 Keeping alternative explanations changes what can be recovered

  • At flip probability .45, WF covered the XOR reference family in 48/50 samples, while the D adapter covered it in 0/50.The joint-information contrast was below the fixed .3-bit protection threshold; the comparison changes upstream search and CI evidence together.Source: §3.2 · p. 8, Figure 3 · p. 8

3.3 Finite evidence limits both recovery and resolution

  • At flip probability .49, WF covered the XOR reference in 12/50 samples, and 36/48 nonempty outputs made a false structural assertion.These were principally false absence claims; the mean incorrectly excluded adjacencies was 2.23 among those 48 outputs.Source: §3.2 · p. 8

3.2 Keeping alternative explanations changes what can be recovered

  • In a faithful four-node chain, WF covered the reference in 42/50 samples, versus 49/50 for eligible PC and the D adapter.WF made false assertions in 7/49 nonempty outputs, compared with 1/50 for each comparator.Source: §3.2 · pp. 8–9

3.4 Time order narrows explanations under suitable assumptions

  • With correct time tiers, mean WF family size fell from 4.82 to 1.00 across independent-baseline episodes, while equality rose from 43/50 to 45/50.For dependent baselines, all 50 tiered outputs and population references were empty, recording incompatibility rather than causal recovery.Source: §3.4 · p. 9

3.5 Increasing size limits useful output and completion

  • Complete nonempty WF outputs fell from 188/200 at four variables to 108/200 at six and 51/200 at eight; all 400 twelve/twenty-variable searches remained unfinished at the reporting cutoff.The six/eight-variable joint-effect outputs contained no planted graph among 200 completed outputs, though 47 were nonempty; above four variables, population-family and assumption classifications were unavailable.Source: §3.5 · p. 9, Table 3 · p. 10

3.6 Neural controls distinguish retention from application success

  • On the fixed neural background, D recovered the complete parent pair in 19/20, 20/20, and 20/20 balanced engineered-XOR controls at 5, 10, and 20 ms, while A–C recovered none.Matching synthetic parent probabilities to selected neural firing proportions made the pair recoverable by every arm, so that regime showed no incremental protection benefit.Source: §3.6 · p. 10
  • All 740 neural family searches remained incomplete at 60 seconds with no accepted WF candidate, leaving local family coverage unavailable.On the observed 18-unit panel, A–D returned identical edge sets within each bin width, with no accepted protection decision or known neural causal truth.Source: §3.6 · p. 10

4 Discussion

The two approaches expose distinct operational requirements: joint relationships must survive deletion, and outputs must retain the alternative explanations allowed by the assumptions. Their gains are conditional on signal and scale, with finite-sample and computational limits constraining broader claims.

4.1 What the operational choices achieve

  • Reactivation and later protection are both needed in the descendant-baseline comparison, because they address distinct deletion stages; selective retention can also coexist with deletion of nuisance candidates.Reactivation keeps a candidate available, while protection prevents a surviving edge from later pruning.Source: §4.1 · p. 11
  • Protection’s value depends on signal, and finite independence decisions can exclude genuine alternatives or leave a completed small family expressing unwarranted certainty.The fixed protection threshold fails below its limit; removing it does not eliminate finite-sample error.Source: §4.1 · p. 11

4.2 Retention and identification require different outputs

  • A selected skeleton cannot restore alternative explanations lost before orientation, while changing assumptions can change which population candidates are admitted.Accuracy, ambiguity, and coverage therefore need joint evaluation rather than a count of returned arrows.Source: §4.2 · p. 11

4.3 The demonstrated scope and the remaining research problem

  • Completed six/eight-variable searches already show frequent empty outputs and poor planted-network recovery in joint-effect scaling cases, while larger searches fail to complete.More runtime is not a demonstrated solution; useful family learning requires progress on inference and computation.Source: §4.3 · p. 11
  • Temporal knowledge can remove backward-arrow explanations only under supported model assumptions, and the neural experiments test one fixed background rather than independent biological replicates.The family estimator did not return a complete graph family within the application’s one-minute search window.Source: §4.3 · pp. 11–12
  • Selected examples and cellwise intervals limit broad performance conclusions, while the findings call for coordinated stages, explicit output meaning, finite-decision uncertainty, and useful scale.Unfinished searches remain inventoried, and observed resource use does not forecast eventual completion.Source: §4.3 · p. 12

5 Conclusion

The paper operationalises weaker faithfulness through coordinated candidate survival and later protection, alongside a distinct search that retains alternative graph explanations. The measured gains come with errors and computational failures, leaving reliable inference at larger scales unresolved.

  • Coordinated candidate survival and later protection recover jointly informative links, while family search preserves alternative graph explanations at the cost of extra edges, false exclusions, and computational failures.Source: §5 · p. 12

A The population target and its construction

The population target makes weaker faithfulness explicit through assumptions on Markov compatibility, adjacency, and subgraph minimality. Its exact-law construction and family representation distinguish retained candidates from a single selected graph.

  • The target assumes finite categorical variables and a strictly positive joint law for affirmative exact-law results; deterministic laws are boundary diagnostics.The graphical Markov blanket is distinct from an algorithm’s current candidate set.Source: A The population target and its… · p. 14
  • The weaker adjacency condition permits an adjacent variable to be supported individually or through a two-variable triple association, evaluated at both edge endpoints.The association group has size at most two; its witness can include a spouse rather than two parents.Source: A The population target and its… · p. 14
  • The principal strictness convention requires failure of both individual one-associations, whereas the alternative requires that they not both hold; this changes the admitted model domain.The example biased-parent XOR law has no principal candidate but four under the alternative convention.Source: A The population target and its… · p. 14
  • Under positivity, each topological order yields a unique inclusion-minimal Markov graph respecting that order, and filtering these graphs gives the exact-law candidate set.The small positive-law suite provides implementation checks, not a finite-sample coverage proof.Source: A The population target and its… · p. 14
  • The principal target does not require the additional orientation-faithfulness predicate, and grouped noncollider evidence remains an existential constraint over actual paths.A consensus drawing does not substitute for the joint candidate constraints.Source: A The population target and its… · p. 14
  • A class representation is accepted only when its full extension set equals the retained candidates; families spanning skeletons require alternatives, and restricted subsets may need residual constraints.The principal population examples establish ambiguity across classes.Source: A The population target and its… · p. 14

B Implementation details for the adjacency learner

The adjacency learner’s implementation fixes traversal and testing choices that shape candidate retention and pruning. Its temporal eligibility rules, CI policy, and protection contrast impose additional operational qualifications.

  • Reactivation uses the first separating set and tests a one-variable extension only when that separator is a singleton; no traversal invariance is claimed.Source: B Implementation details for the… · p. 15
  • Temporal adjacency edges follow implementation-specific eligibility rules, including excluding baseline–baseline edges; these restrictions are not consequences of temporal order alone.Source: B Implementation details for the… · p. 15
  • The primary arms use modified G2 tests at nominal level .05, treating nonrejection as independence without graph-level error control.A separate baseline-only sensitivity varied protection thresholds or applied Benjamini–Hochberg filtering.Source: B Implementation details for the… · p. 15
  • Protected pruning retains ordinary-pruning edges plus protected identifiers that survived earlier removal, assuming a shared pruning-query pool; this does not prove blanket or causal-skeleton recovery.Source: B Implementation details for the… · p. 15

C Finite evidence, uncertainty and comparators

Finite evidence is evaluated through distinct references and testing policies, which answer different questions. Completion of a practical search does not guarantee agreement with an exhaustive finite-decision reference.

Finite testing policies

  • The selected randomization policy uses 199 null draws and makes queries above 262,144 table cells unavailable, while the depth-two G2 family policy treats deeper queries as unavailable.The selected policy has no depth-two restriction; the policies are separate from the protected adjacency learner’s finite decisions.Source: C Finite evidence, uncertainty and… · p. 15

Empirical reference

  • The empirical family exhaustively applies compatibility, association predicates, subgraph minimality, and supplied knowledge to a complete table of Boolean CI decisions.Its four-variable reference does not assume that the table obeys probability-law identities; a completed practical order search can still differ from it.Source: C Finite evidence, uncertainty and… · p. 15

Uncertainty diagnostic

  • The Markov confidence family retains DAGs not separating rejected queries, with simultaneous retention under stated test-validity conditions when no null is falsely rejected.It does not impose weaker adjacency faithfulness or minimality, so its coverage and size address a different question from weaker-faithfulness recovery.Source: C Finite evidence, uncertainty and… · p. 15

C.1 Comparator scope

The comparators have different scopes and cannot be read as interchangeable tests of weaker-faithfulness recovery. Their outputs and eligibility rules limit which comparisons support global family claims.

D adapter

  • For at most four variables, the D adapter enumerates compatible acyclic orientations of the selected skeleton but adds no CI tests or collider evidence.For larger inputs, one returned extension is generally incomplete, so global family assertions are withheld.Source: §C.1 · p. 15

Orientation scores

  • Orientation scores require a correct D skeleton and a nonempty compatible reference restricted to that skeleton, whose size must also be reported.Source: §C.1 · p. 16

PC-stable

  • PC-stable is used only in independently classified faithful four-variable conditions, and its collider orientation can retain order dependence from the first separator.Its reported cost includes exhaustive small-class representation and is not a conventional PC scaling benchmark.Source: §C.1 · p. 16

MD-PC reference

  • The checked MD-PC reference is independently implemented for at most four static variables and accepts compatibility without requiring planted truth.An empty graph is a valid candidate, and a nonempty class excluding truth can pass the compatibility screen; MD and WF family sizes are not interchangeable targets.Source: §C.1 · p. 16

D Design details and full displayed-condition tables

The design varies mechanisms and sample conditions in matched experiments, with saved summaries separating stage and nuisance comparisons. The illustrative trace shows recovery alongside its false-edge cost.

Controlled expansion

  • The 35-condition controlled expansion varies output noise, parent leakage, parent probabilities, and nuisance dimensions across three four-variable baselines.It uses two sample sizes and 20 replicates per condition and size, with shared innovations pairing relevant perturbations.Source: D Design details and full… · p. 16

Panels and trace

  • Figure 2 uses saved expansion summaries for matched stage and nuisance panels, which represent different mechanisms rather than one size series.The illustrative trace is a static descendant input at N = 10,000 and seed 1000; its D output also retains the false X −Y adjacency.Source: D Design details and full… · p. 16

Application design

  • The adjacency application contains 15 observed and 600 engineered executions, and its neural-proportion synthetic parents do not reproduce neural serial dependence.Source: D Design details and full… · p. 16

D.1 Confirmation mechanisms

Confirmation mechanisms and scaling are reported as separate inputs and cells, not as a pooled accuracy estimate. The tables define how family outcomes and scaling completion should be interpreted.

Confirmation design

  • The confirmation comprised 148 cells and 6,400 input IDs; its replication count was reduced before outcomes were generated, and it accounts for 62,300 method executions and 75,700 logical method/view outputs.Each mechanism, parameter setting, sample size, and dimension remains separate in the full exports; Tables 5 and 6 retain every planned sample size for the main family examples and every scaling cell.Source: §D.1 · p. 16

Example definitions

  • Table 4 defines binary-variable examples using independent roots and errors except where stated, and a flip probability e denotes an independent Bernoulli(e) error modulo two.Confirmation parameters differ from those in the controlled expansion.Source: §D.1 · pp. 16–17

Mechanisms

  • Table 4 presents mechanisms behind the reported confirmation examples; for a two-parent conditional-probability table, entries are ordered 00, 01, 10, 11.Source: Table 4 · p. 16

Scaling mechanisms

  • For scaling, parent-selection settings use probabilities .25 and .65 with caps of two and four, and every nonroot has at least one parent.The topology seed includes the full input ID, including N, and above-four-variable weaker-faithfulness classification is not implied.Source: §D.1 · p. 17

Family regimes

  • Table 5 reports four family regimes at planned sample sizes, with 50 inputs per row and all searches complete.Coverage means retaining the entire population WF reference; equality means returning exactly that reference, while false assertions are counted among complete nonempty outputs.Source: Table 5 · p. 17

Scaling accounting

  • Table 6 partitions each 25-input scaling row into complete nonempty, complete empty, and incomplete outputs, while planted membership includes complete empty failures.Above four variables, the full-joint reference and assumption classification are unavailable.Source: Table 6 · p. 18

E Neural sampling and application endpoint

The neural application used sampled transitions from one culture recording, with causal ground truth unknown and key modeling assumptions unverified. At the reporting cutoff, every recorded family-search output was incomplete, while the application had consumed approximately 93,322 CPU seconds.

Neural sampling

  • The analysis used one CRCNS ssc-3 culture recording with 12,000 sampled transition-row positions per bin width, while selected rows could remain dependent.The selected adjacent-source-position counts are not effective sample sizes, and shuffling rows does not establish IID sampling.Source: E Neural sampling and application… · p. 19

Scope and assumptions

  • Positivity, causal sufficiency, stationarity, adequate history, and an acyclic within-bin model remained unverified for the neural population.Observed edges were not labelled biologically correct.Source: E Neural sampling and application… · p. 19

Scope and scoring

  • Synthetic-module references were local operational references, not the full reference family for the combined fifteen/sixteen-variable inputs.Application scores likewise provide neither biological accuracy nor an incremental D-versus-B comparison.Source: E Neural sampling and application… · p. 19, Table 7 · p. 20

Application endpoint

  • Of 4,209 planned method executions, 2,073 had recorded outcomes and 2,136 were unattempted when the active attempt ended.All three observed inputs were reached, but planned 100-replicate denominators were retained.Source: E Neural sampling and application… · p. 19

Family-search endpoint

  • 740 recorded WF outcomes were incomplete at the elapsed search-window limit, with zero accepted DAGs and no evaluable family-recovery or compelled-arrow scores.All-planned binary bounds for those scores remain [0, 1].Source: E Neural sampling and application… · p. 19

Application endpoint

  • Approximately 93,322 CPU seconds were used by recorded application attempts, including monitored child processes.Eight limited D attempts included one storage interruption and seven CPU-limit outcomes.Source: E Neural sampling and application… · p. 20
Loading 2610.31724v1…