Cautious Conflict-Free Inverse Gradient Methods

1 INTRODUCTION

• PINNs parameterize PDE solutions with neural networks, avoiding discretization and easing the curse of dimensionality, but their error analysis, generalization, and training behavior remain less predictable.Source: §1 · p. 1
• PINN task gradients can interfere and hinder convergence; although ConFIG computes a conflict-free direction, subsequent optimizer processing can destroy that property before the weight update.Source: §1 · p. 2
• Cautious ConFIG achieved 19% to 28% relative improvement over the baseline across Burgers, nonlinear Schrödinger, and Kovasznay PINN benchmarks.The benchmarks are the 1D viscous Burgers equation, 1D nonlinear Schrödinger equation, and 2D Kovasznay flow.Source: §1 · p. 2
• Cautious ConFIG is a simple three-line extension compatible with any ConFIG-optimizer combination that restores ConFIG’s mathematical properties with negligible overhead and GPU parallelizability.Source: §1 · p. 2
• A computational model and training metrics indicate Cautious ConFIG aligns update directions more closely with task gradients under severe conflict, numerous tasks, and significant optimizer lag.Source: §1 · p. 2

2 BACKGROUND

• PINNs reformulate initial, boundary, and PDE conditions as m differentiable losses over a neural-network solution, making an IBV problem a multi-task optimization problem.The losses can be differentiated with respect to the network parameters using automatic differentiation.Source: §2 · p. 2
• A gradient conflict is ⟨gi, gj⟩ < 0; scalarization-based updates may minimize one loss while increasing another for sufficiently small η, and Liu et al. (2025) demonstrated PINNs’ susceptibility.Opposing gradients may cancel and slow learning.Source: §2 · pp. 2–3
• ConFIG selects a direction from the intersection of task-gradient half-spaces, giving positive and equal cosine similarities with all task gradients (Liu et al., 2025).Updating along a direction in this intersection decreases all m losses when the learning rate is sufficiently small.Source: §2 · p. 3
• With momentum-based optimizers such as Adam, accumulated history can make the optimizer output conflict with the current gradient; cautious masking removes disagreeing updates and rescales the result (Kingma & Ba, 2014).The resulting cautious update satisfies ⟨ucautious, g⟩ ≥ 0, while rescaling keeps its L1 norm approximately that of the original update.Source: §2 · p. 3

3 RELATED WORK

• PCGrad subtracts gradient projections to remove pairwise conflicts but cannot guarantee a conflict-free summed direction; Nash-MTL uses Nash bargaining, while CAGrad maximizes the minimum task-gradient dot product near the average gradient.Source: §3 · p. 3
• Cautious ConFIG is derived from ConFIG for spatial conflicts and cautious optimizers for temporal conflicts (Liu et al., 2025).The passage identifies adaptive optimizers as the source of the temporal conflicts addressed by cautious updates.Source: §3 · p. 3
• RProp adjusts each parameter’s learning rate according to whether successive gradient signs agree, while Magma skips updates by groupwise cosine similarity and redistributes the masked L1 norm (Joo et al., 2026).RProp increases the rate multiplicatively when signs agree and decreases it otherwise.Source: §3 · pp. 3–4
• GUM addresses the same optimizer-induced conflict, but Cautious ConFIG uses a simpler, metric-invariant orthant projection instead of GUA’s quadratic-program projection onto the conflict-free cone (Xiao et al., 2026).GUA uses an active-set solver and adds optimizer-state alignment with two tunable hyperparameters; Cautious ConFIG uses a parallelizable binary mask, takes three code lines rather than more than 120, and adds no hyperparameters. The work was concurrent, so no direct comparison is provided; both evaluate the Burgers, Schrödinger, and Kovasznay IBV problems.Source: §3 · p. 4

4.1 PROBLEM

• Adam’s momentum and adaptive metric can transform conflict-free gConFIG into a weight-update direction that conflicts with task gradients, invalidating ConFIG’s loss-decrease guarantee.ConFIG’s direction decreases all losses when the learning rate is sufficiently small, but the resulting Adam update need not remain conflict-free.Source: §4.1 · p. 4

4.2 CAUTIOUS CONFIG

• Cautious ConFIG softly aligns Adam’s output toward gConFIG, improving its cosine similarity with that anchor before sign masking and L1-preserving rescaling.The cautious update maintains cosine similarity with gConFIG at least as high as the original optimizer output’s similarity.Source: §4.2 · pp. 5–6
• Masking directly against all task gradients can update only a 2^-m fraction of parameters, destabilizing training, so cautious masking aligns once to summary direction gConFIG.The paper attributes the exponential reduction to requiring all m gradient components to agree in sign and notes that the reduced parameter set receives a proportionally high learning rate.Source: §4.2 · pp. 5–6

• Cautious ConFIG improves relative performance by 19% to 28% over Adam + ConFIG and substantially over plain Adam across three PDE benchmarks.Source: Figure 3 · p. 6, §5 · p. 6
• Evaluation uses minimum training MSE across the Burgers, nonlinear Schrödinger, and Kovasznay PDEs; differing sampling and epoch settings mean absolute MSEs are not directly comparable with prior work.The PINN-specific sampling and training-epoch settings differ from the original ConFIG study because of time and resource constraints.Source: §5 · pp. 6–7
• Across 10 random seeds, Cautious ConFIG reduces both mean MSE and standard deviation, indicating higher performance and greater training stability.Source: §5 · p. 7, Table 1 · p. 7
• For Kovasznay flow, Cautious ConFIG converges more rapidly initially, and its performance gap grows as the task count increases from m = 2 to 4.Source: §5 · p. 7

6 ANALYSIS

• Cautious masking skips sign-conflicting coordinates and rescales the remaining updates, redistributing update energy toward non-conflicting coordinates.The analysis also reports fewer task-gradient conflicts along cautious trajectories and a greater-magnitude gConFIG input to Adam; Figures 5 and 6 provide supporting evidence.Source: §6 · p. 7, Figure 5 · p. 9, Figure 6 · p. 10
• Cautious ConFIG always increases cosine similarity to gConFIG and empirically improves similarity to every task gradient over the unmodified optimizer output.Figure 4 confirms the task-gradient comparison.Source: §6 · p. 7, Figure 4 · p. 9

CONFLICT MODEL

• The conflict model builds high-dimensional task gradients around a shared conflict axis, with conflict strength interpolating between opposed gradients and Gaussian noise.At c = 1, task gradients split into oppositely oriented groups; at c = 0, the model recovers independent standard-normal gradients.Source: CONFLICT MODEL · p. 8

OPTIMIZER MODEL

• The optimizer analysis models output as gConFIG plus isotropic noise and applies cautious masking, but omits internal optimizer state and preconditioning.High noise represents optimizer lag for momentum-based optimizers; the analysis notes that inverse-metric preconditioning distorts the space and is not captured.Source: OPTIMIZER MODEL · p. 11

QUALITATIVE ANALYSIS

• As conflict strength and task count rise, cautious updates retain task-gradient alignment and conflict-freeness better than unmodified optimizer outputs.The unmodified output’s conflict-free probability falls sharply at high conflict strength, while the cautious output remains nearly 100% conflict-free until conflict is maximal.Source: QUALITATIVE ANALYSIS · p. 11, Figure 9 · p. 11
• Under greater optimizer noise, cautious updates preserve common-descent probability and positive task-gradient alignment; increasing model size further boosts cautious conflict-freeness and alignment.The simplified model finds that cautious conflict-freeness rises sharply around d ≈200 and remains near 100% at larger sizes, while average cosine similarity plateaus above 15%.Source: QUALITATIVE ANALYSIS · pp. 12–13, Figure 10 · p. 12, Figure 11 · p. 12

7 CONCLUSION

• 19% to 28% relative improvement over the baseline is reported for Cautious ConFIG across the Burgers, Schrödinger, and Kovasznay PINN scenarios.The method’s modified optimizer output is empirically conflict-free with respect to task gradients before the parameter update.Source: §7 · p. 13
• Cautious ConFIG is a parallelizable three-line extension with negligible computational overhead and no additional hyperparameters.The extension addresses the loss of ConFIG’s mathematical guarantees when an adaptive optimizer such as Adam processes its direction.Source: §7 · p. 13
• The toy model shows improved task alignment under significant optimizer lag, numerous tasks, and pronounced gradient conflicts.Source: §7 · p. 13
• Future work could integrate cautious updates with PCGrad or Nash-MTL, or apply the method to physics-constrained generative models (Baldan et al. (2026)) and hybrid numerical methods (Franz et al. (2025)).Source: §7 · p. 13
• The current work is limited to computational experiments; future analysis could derive toy-model relationships involving task number and conflict strength or develop a more realistic, analytically tractable optimizer model.The proposed improved toy system would incorporate strong conflicts in high-dimensional spaces, as observed in the PINN setting.Source: §7 · p. 13

PROPOSITION (CAUTIOUS MASKING INCREASES COSINE SIMILARITY WITH CONFIG DIRECTION)

• Cautious masking makes the direction more aligned with gConFIG than its unprocessed counterpart by increasing the numerator and decreasing the cosine-similarity denominator.Cosine similarity is invariant to scaling, and zeroing components cannot increase the masked vector’s length.Source: PROPOSITION (CAUTIOUS MASKING… · pp. 14–15

PROPOSITION (CAUTIOUS MASKING CREATES NON-NEGATIVE DOT PRODUCTS)

• With ϕ = 1{u ⊙ g > 0}, cautious masking ensures ⟨u ⊙ ϕ, g⟩ ≥ 0.Source: PROPOSITION (CAUTIOUS MASKING CREATES… · p. 15

• The Burgers benchmark seeks a fluid-velocity solution on [−1, 1] × [0, 1], with initial and boundary conditions specified.The passage does not provide the condition values in the supplied text.Source: EXPERIMENTAL DETAILS · p. 15
• The nonlinear Schrödinger setup uses x ∈ [−5, 5] and t ∈ [0, π/2], periodic boundaries, and h(x, 0) = 2sech(x).Source: EXPERIMENTAL DETAILS · p. 15
• The Kovasznay-flow setup specifies its PDE and uses the analytical solution for the Dirichlet boundary condition.Source: EXPERIMENTAL DETAILS · pp. 15–16
• Shared and PINN-specific hyperparameters are presented separately in Tables 3 and 4.The supplied captions do not expose their parameter values.Source: Table 3 · p. 16, Table 4 · p. 16