GFlowNet Foundations

1.1 What is a GFlowNet ?

• GFlowNets construct compositional objects through stochastic steps, learning sampling probabilities approximately proportional to non-negative rewards R(s), while flow-matching constraints conserve incoming and outgoing flow.After training, they generate samples in one pass rather than through the lengthy stochastic search required by MCMC, provided the reward structure supports generalization across modes.Source: §1.1 · pp. 2–3

1.2 Contributions of this paper

• The paper introduces conditional GFlowNets for estimating marginal probabilities, free energies, entropy, conditional entropy, and mutual information over sets, graphs, and other composite outcomes.It also proposes detailed balance, unsupervised Pareto-frontier sampling, intermediate-reward extensions, stochastic and continuous-environment formulations, and jointly learned modular energy functions.Source: §1.2 · pp. 3–5
• GFlowNets support amortized probabilistic inference, including posterior sampling when rewards combine priors with likelihoods, and can be trained offline while matching reward distributions rather than maximizing reward.These properties connect GFlowNets to amortized variational inference and support diverse exploration of target-distribution modes.Source: §1.1 · p. 2, §1.2 · pp. 3–4

1.3 GFlowNets in other works

• Prior work applies GFlowNets to proxy-based search, active learning, Pareto-optimal generation, energy-based modeling, causal-graph structure learning, and latent-variable inference, with reported advantages in diversity and some off-policy settings.Applications include molecules, DNA, proteins, computation schedules, binary vectors, and causal graphs; related theory establishes partial equivalences with hierarchical variational methods (Zimmermann et al., 2022).Source: §1.3 · pp. 5–6

2.1 Some elements of graph theory

• Pointed DAGs provide unique source and sink states, complete trajectories from s0 to sf, and terminating states connected to the sink.The single-source/single-sink requirement is a mathematical convenience, and terminating states are Par(sf).Source: §2.1 · pp. 6–7

2.2 Trajectories and Flows

• A flow assigns non-negative mass to complete trajectories and induces corresponding flows through states and edges, with state flow decomposing over outgoing edges and edge flow over parent states.Together, these definitions make (G, F) a flow network and establish the conservation relations between state and edge flows.Source: §2.2 · p. 9

2.3 Flow Induced Probability Measures

• Flow normalization yields trajectory and transition probabilities, while terminating-state probabilities form a distribution proportional to terminating flows with normalization Z = F(s0).The initial-state flow, final-state flow, and total flow coincide, and PT is well-defined over Sf even though P(s) is not generally a state distribution.Source: §2.3 · pp. 10–11

2.4 Markovian Flows

• Markovian flows depend on the current state rather than trajectory history, equivalently factorizing complete-trajectory probabilities through unique graph-consistent forward and backward transition functions.This representation replaces the generally exponential specification of one flow value per trajectory with substantially fewer quantities, while allowing later state-conditional memory variants.Source: §2.4 · p. 12
• Markovian flows are uniquely specified by total flow with forward probabilities, total flow with backward probabilities, or terminating flows with backward probabilities on non-terminating edges.Sampling forward transitions until termination produces terminating state s with probability PT(s).Source: §2.4 · p. 16

2.5 Flow Matching Conditions

• A non-negative state- or edge-flow function uniquely defines a Markovian flow exactly when it satisfies the flow-matching conditions.Equivalently, state flows and compatible forward/backward transitions define a unique flow when detailed balance holds: ˆF(s) ˆPF(s′ | s) = ˆF(s′) ˆPB(s | s′).Source: §2.5 · pp. 18–19

2.6 Backwards Transitions can be Chosen Freely

• Terminating flows generally do not determine a flow completely, so backward transitions on non-terminal edges can freely encode preferences among alternative paths to the same outcome.They may weight parents equally, favor shorter paths, or be learned to simplify forward-policy or flow learning.Source: §2.6 · p. 21

2.7 Equivalence Between Flows

• Equivalence by coinciding edge flows partitions all flows into classes, with exactly one Markovian representative per class; equivalent Markovian flows are identical.The figure illustrates equivalent and non-equivalent flows, including cases sharing terminating flows while differing in Markovianity.Source: §2.7 · pp. 21–22, Figure 4 · p. 22
• Each equivalence class contains a unique Markovian flow that agrees with its representative on state and edge flows.The Markovian flow is distinguished from other flows in the same equivalence class by this property.Source: §2.7 · p. 22
• Because only state and edge flows matter, flow-learning problems can be restricted to Markovian flows.This preserves the relevant state and edge values while avoiding the full set of flows.Source: §2.7 · p. 22
• Markovian flows replace trajectory-level specifications with edge-flow specifications, which are generally exponentially smaller and must satisfy flow-matching conditions.A general flow specifies F(τ) for every trajectory, whereas a Markovian flow specifies F(s → s′) for every edge.Source: §2.7 · p. 23

3 GFlowNets: Learning a Flow

• GFlowNets estimate Markovian flows matching a target reward and generate complete trajectories by sampling learned forward transitions until the sink.They use estimators of state flows or transition probabilities, which may initially fail to define a proper flow.Source: §3 · p. 23

3.1 GFlowNets as an Alternative to MCMC Sampling

• GFlowNets amortize the cost of sampling by training a generator that constructs one configuration per sample, avoiding potentially long MCMC chains and mode-mixing.This advantage depends on exploitable structure among reward modes; randomly placed modes can make training intractable in high-dimensional spaces.Source: §3.1 · pp. 23–24

3.2 GFlowNets and flow-matching losses

• A GFlowNet is formally defined by a pointed DAG, reward function, and flow parametrization whose valid configurations induce the target reward-proportional terminating-state distribution.Edge-flow, forward-transition, and forward/backward-transition parametrizations are each shown to preserve the probability measure associated with a Markovian flow.Source: §3.2 · pp. 24–26
• Flow-matching losses are nonnegative objectives whose zeros exactly represent valid flow configurations, enabling gradient-based learning through edge-, state-, or trajectory-decomposable forms.The detailed-balance and trajectory-balance losses instantiate edge- and trajectory-decomposable alternatives, while stochastic gradients use sampled edges or trajectories from a training distribution.Source: §3.2 · pp. 26–29

3.3.1 Introducing Time Stamps to Allow Cycles

• Time-stamping states as (s_t, t) converts cyclic underlying transitions into an augmented DAG and can bias backward transitions toward shorter paths.The construction can replace natural-number timestamps with any totally ordered indexing set.Source: §3.3.1 · p. 29

3.3.2 Stochastic Rewards

• With stochastic rewards, trajectory-balance training optimizes the squared loss using the conditional expectation of log R(s).This follows from minimizing the expected squared-error loss with a stochastic target.Source: §3.3.2 · p. 30

3.3.3 GFlowNets can be trained offline

• GFlowNets can be trained offline from any full-support training distribution rather than their own trajectory distribution.Adaptive sampling can improve efficiency by continually generating new training trajectories, except that rare high-reward trajectories may warrant rehearsal.Source: §3.3.3 · p. 30
• Offline training distributions should adaptively explore under-visited regions while revisiting known high-reward trajectories to avoid missing or forgetting reward modes.Possible strategies include uncertainty-guided exploration or a second GFlowNet targeting states where the main model’s losses are large.Source: §3.3.3 · p. 30

3.4 Exploiting Data as Known Terminating States

• Known terminating states can train flows and forward transitions by sampling backward trajectories, but dataset-only trajectories lack full support and may yield an incorrect sampling distribution.Combining dataset trajectories with a training distribution covering all possible trajectories guarantees recovery of a flow-matching model.Source: §3.4 · pp. 30–31

4 Conditional Flows and Free energies

• The initial flow F(s0) recovers the normalizing constant Z, while free energies provide a generic formulation for marginalizing many energy-function terms.Estimating free energies can replace otherwise expensive MCMC-based marginalization.Source: §4 · pp. 31–32

4.3 State-conditional flow networks

• State-conditional flow networks repair the failure of ordinary state flows to marginalize descendant rewards by defining flows on subgraphs anchored at each state.For every state s, the initial conditional flow marginalizes terminal flows F(s′→sf) for s′ ≥s and equals the associated free energy.Source: §4 · p. 31, Figure 5 · p. 31, §4.3 · pp. 34–35

4.1 Conditional flow networks

• Conditional flow networks form families of DAG-indexed flows whose probabilities, transitions, Markovian properties, and flow-matching conditions inherit the ordinary definitions for each conditioning value x.The conditioning information may be external or internal to the flow network.Source: §4.1 · pp. 32–33

4.2 Reward-conditional flow networks

• Reward-conditional flow networks keep the DAG fixed while conditioning only terminating flows on x, including energy-based models with reward R(s | θ) = exp(−Eθ(s)).Conditional GFlowNets estimate these networks by making learned flows and training losses depend on x.Source: §4.2 · p. 33, §4.4 · p. 36

4.3 State-conditional flow networks

• State-conditional flows also induce probabilities of terminating in descendant states under redirected terminal flows, enabling conditional distributions over states s′ ≥s.This provides a probabilistic interpretation of the state-anchored marginalization construction.Source: §4.3 · p. 35

4.5 Training Energy-Based Models with a GFlowNet

• A conditional GFlowNet can approximate sampling from an energy-based model by targeting R(s) = e−Eθ(s), yielding an unbiased gradient estimator when its loss is zero.The energy model and sampler can be jointly trained, although the required number of inner GFlowNet updates remains an open question.Source: §4.5 · pp. 36–38

4.6 Active Learning with a GFlowNet

• In active learning, GFlowNets train on a learned proxy reward and can combine predicted usefulness with epistemic uncertainty to explore diverse, informative candidates.Experiments reported here show greater solution diversity than PPO, which tends to focus on one reward mode.Source: §4.6 · p. 38

4.7 Estimating Entropies, Conditional Entropies and Mutual Information

• Training one GFlowNet on R and another on its entropic reward allows initial flows to estimate entropy, conditional entropy, and mutual information.The construction applies the same marginal-flow principle to conditional flows.Source: §4.7 · p. 39
• Conditional flow networks estimate H[S | x] from the conditional flow F(s0 | x) and its entropic counterpart, provided R_x(s) < 1 for all s.The positivity condition ensures the transformed rewards and flows remain positive.Source: §4.7 · p. 39
• A state-conditional GFlowNet yields a corresponding entropy estimate when the conditioning variable is the state space of the DAG.This is the specialization X = S of the conditional-entropy construction.Source: §4.7 · p. 40
• Mutual information MI(S; X) is estimated by combining unconditional flows with their conditioned counterparts for samples of the conditioning variable X.The formulation distinguishes F(s) and F′(s) from F(s | x) and F′(s | x).Source: §4.7 · p. 40
• When sampling from P(X) is available, the expectation in the mutual-information estimator can be approximated by a Monte Carlo average.The average uses draws from P(X).Source: §4.7 · p. 40

5.1 Set GFlowNets

• GFlowNets trained on set rewards can represent and sample set distributions while estimating set probabilities, partition functions, conditional and marginal superset probabilities, lowest-energy sets, and entropies.Set GFlowNets construct subsets by adding elements from a universe U; approximate flow estimates may not produce probabilities that exactly sum to 1.Source: §5.1 · pp. 41–42

5.2 GFlowNet on Graphs

• Graph GFlowNets specialize set construction to add nodes and edges under constraints such as connectivity or acyclicity, accurately approximating posterior distributions over Bayesian-network DAGs.Because graphs are sets, the set-based GFlowNet operations apply directly to graphs.Source: §5.2 · p. 42

5.3 Marginalizing over Missing Variables

• GFlowNets can model joint distributions over composite variables and estimate or sample conditionals and marginals by representing variable assignments as sets of indexed value pairs.The construction permits variables to be sampled in any policy-allowed order, with missing variables marginalized implicitly or sampled by continuing the policy.Source: §5.3 · p. 42
• For joint-variable GFlowNets, the same framework supports partition-function estimation, probable ordering, maximum-probability configuration search, and entropy estimation.These operations remain feasible when only a subset of variables is specified and the others are marginalized or sampled conditionally.Source: §5.3 · pp. 42–43

5.4 Modular Energy Function Decomposition

• Modular factor-graph GFlowNets represent reusable energy terms as graph pieces, enabling sampling of latent structures conditioned on observations and decomposition of both energies and GFlowNet policies into mechanism-specific modules.Observed variables can be marginalized over latent graph specifications to compute free energies; structural priors and type constraints can guide graph construction, and Forward-Looking GFlowNets improve target-distribution approximation (Pan et al. (2023)).Source: §5.4 · pp. 43–44

6 Continuous or Hybrid Actions and States

• The discrete-state framework extends to continuous or hybrid states and actions by replacing sums with integrals, while continuous transitions require tractable density evaluation and sampling.Normalization constants are implicit in successor and predecessor conditionals, creating a central challenge for detailed balance and related constraints.Source: §6 · p. 44

6.1 Integrable Normalization Constants

• Hybrid transitions can separate discrete choices from continuous choices, using tractable densities, mixtures, autoregressive or normalizing-flow models, or diffusion-style resampling.A joint edge-flow parameterization over discrete and continuous components makes forward and backward conditional densities exactly computable and compatible.Source: §6.1 · pp. 44–45

6.2 GFlowNets in GFlowNets

• Nested ⟨GFlowNet, energy function⟩ modules can represent continuous edge flows, sample transitions in either direction, and evaluate their normalizing constants and conditional probabilities.Lahlou et al. (2023) extended GFlowNet theory to general state spaces, including continuous ones, and experimentally validated usual losses for continuous-domain inference.Source: §6.2 · pp. 45–46

7.1 Contrast with Generative Models

• Unlike VAEs and GANs trained on finite examples, GFlowNets are normally trained from energy or reward functions that distinguish likely, unlikely, and intermediate candidates.GFlowNets are designed for discrete, variable-size compositional structures, whereas standard generative models begin with fixed-size real-valued vectors and latent variables.Source: §7.1 · pp. 46–47

7.2 Contrast with Regularized Reinforcement Learning

• GFlowNets target P_T(s) ∝ R(s), whereas MaxEnt RL targets P_T(s) ∝ n(s)R(s), over-weighting states with many trajectories unless the DAG is a tree.This distinction motivates GFlowNets when sampling terminating states in proportion to reward is the objective.Source: §7.2 · p. 48

7.3 Contrast with Monte-Carlo Markov Chain methods

• GFlowNets replace MCMC’s long iterative chains with amortized sampling, potentially jumping between distant modes when learned structure supports generalization.MCMC mode mixing can require exponentially long chains, while GFlowNets shift cost toward training and make each subsequent sample inexpensive; MCMC may remain preferable for a single sample.Source: §7.3 · p. 49
• Compared with SMC, GFlowNets require no intermediate targets or per-step likelihood reweighting, and their trajectories are defined by terminal rewards even when intermediate states are not valid samples.Evolutionary methods obtain diversity through populations and group metrics, whereas GFlowNets use i.i.d. sampling with nonzero probability for suboptimal candidates.Source: §7.3 · p. 50

Core framework

• Detailed balance separates backward-policy preferences over action order from constraints imposed by the target reward, extending GFlowNets’ mathematical framework and trajectory-flow interpretation.The paper connects GFlowNet flow with measures over trajectories and introduces detailed balance as a novel training objective.Source: §8 · p. 50

Marginalization and free energy

• Conditioned GFlowNets generalize partition-function estimation to intractable reward sums over reachable terminating states, enabling supergraph, superset, entropy, conditional-entropy and mutual-information estimation.This framework applies to supergraphs, supersets of sets, and supersets of variable-value pairs.Source: §8 · p. 50

Direct credit assignment

• Alternative objectives explore direct credit assignment to reduce slow end-to-start bootstrapping, alongside trajectory balance and related future work.The direct-credit-assignment alternatives have some similarity to policy gradient.Source: §8 · p. 50

Connections to reinforcement learning

• Reinforcement-learning connections cover rewards during trajectories and stochastic environments, including greedy return-maximizing policies and value-like downstream-reward functions while retaining sampling proportional to return.These developments span Secs. C–E and introduce a notion of return for rewards earned along trajectories.Source: §8 · pp. 50–51

Multiple flows and Pareto sampling

• Learning families of flows over one graph enables distributional and unsupervised GFlowNets, including post-training reward specification and sampling from Pareto fronts.The distributional analogy is to distributional reinforcement learning.Source: §8 · p. 51

Open questions

• Open questions include continuous actions and states, hierarchical abstract actions, and integrating energy functions into parametrizations, while empirical validation is needed for algorithms and applications involving MCMC, probabilistic reasoning and scientific discovery.The proposed applications include replacing or combining with MCMC and extending active learning for scientific discovery.Source: §8 · p. 51

Appendix A. Direct Credit Assignment in GFlowNets

• Direct credit assignment can propagate reward-mismatch information to earlier transitions more quickly than flow-matching or detailed-balance updates, especially on long trajectories.Trajectory Balance is identified as an alternative direct training objective.Source: Appendix A · p. 57
• Flow perturbations propagate through matched networks according to forward and backward conditional trajectory probabilities, enabling long-range equilibrium-gradient estimators.These identities are exact only in the limit of matched flows and become biased for imperfectly trained GFlowNets.Source: Appendix A · pp. 57–58
• The proposed total-derivative estimator is unbiased in the matched-flow limit, and convex combinations of its forward- and backward-transition estimators remain asymptotically unbiased.The endpoint cases correspond to λ = 1 and λ = 0, while any 0 ≤ λ ≤ 1 is covered by convex combination.Source: Appendix A · pp. 59–61
• A policy-gradient-like estimator supplies earlier training signals, but it is exactly valid only online under the learner’s current policy; off-policy trajectories require importance weighting and can introduce variance.Sampling from a broader training distribution can be corrected using trajectory-probability ratios, though numerical variance problems remain.Source: Appendix A · pp. 61–62
• Using slightly mismatched or replay-buffer sampling may accelerate initial training, but it can introduce bias and higher variance, while local flow-matching gradients remain necessary for unbiasedness.Continuous-state credit assignment may additionally require differentiable transitions or backpropagation through time.Source: Appendix A · p. 62

Appendix B. Conditional GFlowNets

• Conditional GFlowNets represent families of DAGs, rewards, flow parametrizations, and policies indexed by conditioning information x.A configuration contains one component per condition, forming a product-space conditional configuration.Source: Appendix B · p. 62
• For each condition x, a flow-matched component induces terminating-state probabilities proportional to the target reward Rx.Conditional GFlowNets therefore cast simultaneous sampling as searching for configurations whose components are flow-matched.Source: Appendix B · p. 63
• Conditional flow-matching losses can decompose across conditions and, in particular, be chosen as state-, edge-, or trajectory-decomposable.The construction starts from any family of per-condition flow-matching losses.Source: Appendix B · p. 63
• Shared amortized parameters can learn flows jointly across conditions, enabling generalization over both conditions and edges.With finite capacity, optimization may trade off performance across conditions because the global zero-loss optimum can be unattainable.Source: Appendix B · pp. 63–64

Appendix C. Policies in Deterministic and Stochastic Environments

• The policy extension separates action selection from environment transitions, allowing the environment to transform actions into next states deterministically or stochastically.Even states transition to odd state-action states through the policy, while the environment governs odd-to-even transitions.Source: Appendix C · pp. 64–65
• In this extension, backward detailed balance also splits into policy inversion and learned inversion of the environment transition.A normalized backward conditional distribution preserves path preferences while guaranteeing flow matching through detailed balance.Source: Appendix C · p. 65

C.1 Known Deterministic Environments

• Known deterministic environments reduce to standard GFlowNet transitions: PF(st+1|st) is obtained by marginalizing the policy over actions satisfying st+1 = T(st,at).When one action explains a transition, the marginalization contains a single term.Source: §C.1 · pp. 65–66
• In deterministic environments, backward transition probabilities can likewise be derived from a backward policy, with a single explaining action yielding the corresponding special case.The backward construction follows the same reasoning as the forward transition result.Source: §C.1 · p. 66

C.2 Unknown Deterministic Environments

• Unknown deterministic environments require learning T and T^−1, with T^−1(T(s, a), a) = s recovering the previous state.The inverse transition maps a next state and action back to the preceding state.Source: §C.2 · p. 66
• In high-dimensional discrete state and action spaces, learning transitions that generalize to unseen cases may be difficult.Recorded transitions cover only an exponentially tiny fraction of future transitions in combinatorial state spaces.Source: §C.2 · p. 66
• Continuous relaxation or methods for stochastic environments may provide more achievable approaches to learning the required transition models.These alternatives are suggested specifically for improving generalization beyond seen transitions.Source: §C.2 · p. 66

C.3 Stochastic Environments

• Any policy can yield a Markovian flow in a stochastic environment, but perfectly achieving desired terminal flows R(s) may be impossible.With sufficient capacity and training time, sequentially defined even- and odd-step transitions produce a Markovian flow.Source: §C.3 · p. 67
• Unknown environment transitions can be estimated from observed (st, at, st+1) triplets, but learned backward transitions depend on the policy and may remain imperfectly compatible.Finite capacity and training time can prevent exact flow matching even in completely controllable environments.Source: §C.3 · pp. 67–68
• If a terminal state has positive reward but zero-probability incoming transitions, no policy can supply its desired flow.Figure 6 gives a two-path counterexample where T(s′′′|s′′, a′′) = 0 forces PB(s′′, a′′|s′′′) = 0, preventing free backward-transition choice.Source: §C.3 · pp. 67–68, Figure 6 · p. 68
• Environment transitions constrain backward transitions, so PB cannot generally be chosen freely while matching flows and terminal rewards.Even-to-odd backward transitions are deterministic, whereas odd-to-even forward transitions are fixed by P(st→st+1|st, at).Source: §C.3 · p. 68

• Expected reward is defined for any distribution P over terminating states, including distributions induced by arbitrary GFlowNet policies.The quantity is introduced for a terminal reward function R after visiting state s.Source: Appendix D · pp. 68–69
• Under zero intermediate rewards and discount factor γ = 1, a policy greedy with respect to V_Pπ achieves expected reward no worse than π.The expected-reward function therefore supports a policy-improvement theorem analogous to value-function improvement in reinforcement learning.Source: Appendix D · p. 69
• A policy π* exists that maximizes expected reward for every state, obtained by greedily improving the policy whose terminal distribution is PT(s) = R(s)/Z.This uses the GFlowNet policy associated with the target reward-proportional terminal distribution.Source: Appendix D · p. 70
• Expected reward under PT can be estimated by training a second flow, or additional GFlowNet heads, with R2 as its reward function.For flows matching R and R2, the resulting expression relates the expected reward under F’s terminal distribution to self-flows from the two flows.Source: Appendix D · p. 70

D.1 Preference for High-Reward Early Trajectory

• A backward policy PB that assigns more probability to state-action pairs with larger V ((st, at)) makes GFlowNet trajectories prefer those construction orders.This preference can be used to favor starting states with larger expected reward over their potential continuations, as established using Theorem 51.Source: §D.1 · p. 70

Definitions

• Trajectory returns sum intermediate rewards, while expected future returns average downstream returns over trajectories conditioned on visiting a state.The expectation uses the flow’s probability measure over trajectories.Source: Appendix E · pp. 70–71

Return recursion

• Expected future returns satisfy a recursion analogous to the undiscounted episodic Bellman equation, but flows propagate through backward rather than forward transitions.The return recursion propagates values backward using forward transitions, whereas the state-flow recursion propagates flows using backward transitions.Source: Appendix E · p. 71

Return-augmented GFlowNets

• A return-augmented GFlowNet stores accumulated environment reward ν(s) in each state so the terminal reward can equal that accumulated return.This construction also accommodates stochastic environment rewards because the accumulated reward becomes part of the state.Source: Appendix E · pp. 71–72
• After complete training, sampling from a return-augmented GFlowNet produces accumulated rewards ν with probability proportional to ν.Such a GFlowNet can only be trained offline from terminated trajectories whose returns have been observed.Source: Appendix E · p. 72

Distributional GFlowNets

• Distributional GFlowNets condition on complete-trajectory returns, using return-augmented terminating states to represent different environmental outcomes and their statistics.For few outcomes, separate output heads can track each label; more generally, the outcome is supplied as a conditional input.Source: Appendix F · pp. 72–73

Outcome-conditioned GFlowNets

• Outcome-conditioned GFlowNets condition flows on a known state-derived outcome y=f(s), including reward values or feature vectors.The outcome may be discrete for simplicity, while continuous-outcome generalization is anticipated.Source: Appendix F · pp. 72–73
• A fully trained outcome-conditioned GFlowNet samples terminating states guaranteed to satisfy the chosen outcome, analogous in practice to goal-conditioned or reward-conditioned policies (Ghosh et al., 2018; Schmidhuber, 2019).This provides a principled route to sampling objects with high reward under a specified reward function, despite imperfect training in practice.Source: Appendix F · p. 73

Distributional GFlowNets

• Training is offline because the conditioning outcome may be known only after trajectory sampling, using positive unconditional trajectories and conditional samples with outcome-matching targets.The proposed procedure samples τ+ unconditionally, conditions on y+=f(s+), updates with target reward 1, then samples τ− conditionally and uses target 1_{y−=y+}.Source: Appendix F · p. 73

Limitations and future work

• Imperfect conditional training becomes unlikely to reproduce the requested outcome when many outcomes exist, motivating smoother conditional rewards and future extensions to stochastic rewards or environments.The sharp indicator reward is sparse, while stochastic-environment extensions remain an open question.Source: Appendix F · p. 73

A posteriori reward definition

• An outcome-conditioned GFlowNet can be transformed into one matching a posteriori reward R = r∘f by combining its conditional flows, without retraining.The construction applies when terminal outcomes are y = f(s) and the desired reward is specified as r(y) after training.Source: §F.1 · p. 74
• The resulting policy πr∘f(a|s) is obtained from outcome-conditioned state flows, state-action flows, and action policies.The transformation uses F(s|y), F((s, a)|y), and π(a|s, y).Source: §F.1 · p. 74
• The conversion works for arbitrary outcome rewards because event probabilities conditioned on terminal trajectories depend on the backward policy, not the reward choice.The construction is validated in the indicator-reward case and extended to general r through the flow identities.Source: §F.1 · p. 74

Runtime conversion

• At runtime, specialization requires summing over the outcome space for each action, with Monte Carlo integration introducing a computation–accuracy trade-off.The pre-trained network can support probabilities and sampling for all possible outcomes, but decisions become more expensive as Monte Carlo sample counts change.Source: §F.1 · p. 75

Reward functions

• Pareto GFlowNets define additive and multiplicative terminal rewards from convex weights over multiple objective outcomes.The objectives are represented as y_i = f_i(s), with weights drawn from a discrete set W.Source: §F.2 · p. 75

Conditional training

• Conditioning a GFlowNet on ω enables scanning weight settings to obtain different policies, predicted probabilities, or free energies.The conditional terminal reward is R_ω(s) = R(s|ω).Source: §F.2 · p. 75

Generalized objectives

• The approach generalizes to non-convex and non-linear objective combinations whenever the combined objective is parameterized by ω.This yields a family of GFlowNets indexed by loss-function variants.Source: §F.2 · p. 76

Pareto-frontier sampling

• Sampling convex weights followed by trajectories draws diverse solutions from the Pareto frontier for multi-objective optimization and sampling.Different weights correspond to different trade-off points among objectives.Source: §F.2 · p. 76

Outcome-driven GFlowNets

• An outcome-driven GFlowNet can condition on objective values y and use basis expansions to disentangle objectives, potentially improving generalization.This is advantageous when objectives have distinct modeling properties, such as one being stationary and another non-stationary.Source: §F.2 · p. 76