PaperlayerView exact revision on arXiv

Source-linked AI summary

GFlowNet Foundations

Yoshua Bengio, Salem Lahlou, Tristan Deleu, Edward J. Hu, Mo Tiwari, Emmanuel Bengio

arXiv:2111.09266v5cs.LGcs.AIstat.ML

TL;DR

GFlowNets address the computational cost and mode-mixing challenges of sampling reward- or energy-based distributions with MCMC. The paper develops their mathematical framework, establishes reward-proportional terminal-state sampling, and extends GFlowNets to marginalization, free-energy estimation, and related distributional quantities.

  • Problem

    MCMC sampling from energy-based distributions can require long chains and suffer from mode mixing, motivating a more efficient sampling approach.

  • Method

    The paper models sequential constructions as flows over trajectories, using Markovian factorization and freely chosen backward transitions to parameterize GFlowNets.

  • Results

    The framework establishes that forward sampling terminates according to the target terminal-state measure proportional to reward and supports exact flow-probability correspondence.

  • Takeaways & Limitations

    GFlowNets can in principle estimate partition functions, free energies, marginal distributions over supersets and supergraphs, entropies, conditional entropies, and mutual information.

  • Takeaways & Limitations

    Using a GFlowNet to train an energy-based model remains an open question whose success likely depends on the underlying data-generating distribution.

Abstract

from arXiv · show

Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them approximately sample in proportion to a given reward function. In this paper, we show a number of additional theoretical properties of GFlowNets. They can be used to estimate joint probability distributions and the corresponding marginal distributions where some variables are unspecified and, of particular interest, can represent distributions over composite objects like sets and graphs. GFlowNets amortize the work typically done by computationally expensive MCMC methods in a single but trained generative pass. They could also be used to estimate partition functions and free energies, conditional probabilities of supersets (supergraphs) given a subset (subgraph), as well as marginal distributions over all supersets (supergraphs) of a given set (graph). We introduce variations enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.

1 Introduction

GFlowNets learn constructive policies that sample composite objects approximately in proportion to reward, while amortizing expensive probabilistic computation. This paper develops theory for marginalization, richer inference, and broader environments and objectives.

  • What is a GFlowNet ?: Their sampling policy is trained so an object’s probability is approximately proportional to its reward, supporting diverse multimodal distributions over sets, graphs, and other composite objects.The reward can be expressed through an energy function E(s) = −log R(s).
  • What is a GFlowNet ?: Compared with MCMC, GFlowNets shift computation into training and can generate each new configuration in a single constructive pass rather than a potentially long chain.MCMC may suffer mode mixing when modes are separated by long sequences of low-probability configurations.
  • What is a GFlowNet ?: GFlowNets construct objects through stochastic sequences from an initial state, evaluating reward only when reaching a terminating state.Flows propagate through intermediate states while satisfying incoming-equals-outgoing flow constraints.
  • Contributions of this paper: Conditional GFlowNets estimate intractable marginal sums and free energies over joint distributions, including supersets, supergraphs, entropies, conditional entropies, and mutual information.The framework conditions the GFlowNet to estimate rewards over terminal states reachable from an arbitrary state.
  • Contributions of this paper: GFlowNets also support offline training and match rewards in probability rather than seeking only a reward-maximizing configuration.The paper relates these properties to amortized probabilistic inference and Bayesian posterior sampling when rewards combine priors and likelihoods.
  • Contributions of this paper: The paper introduces detailed balance, unsupervised Pareto-frontier sampling, intermediate-reward extensions, and formulations for stochastic environments, continuous actions, and learned modular energy functions.Detailed balance avoids explicit sums required by the previously proposed flow-matching loss.

2 Flow Networks and Markovian Flows

This section defines pointed DAGs, flow-induced probabilities, and Markovian flows, then characterizes their equivalent representations and detailed-balance conditions. Markovian flows preserve the relevant state and edge flows while replacing trajectory-level specifications with edge-level ones.

  • Graph definitions: A pointed DAG has unique source state s0 and sink state sf, with every other state reachable from s0 and able to reach sf.Complete trajectories begin at s0 and end at sf; states directly preceding sf are terminating states, and their transitions to sf are terminating edges.
  • Flow-induced probability measures: A flow network induces a probability distribution over terminating states proportional to terminating flows, normalized by the initial flow Z.This distribution is denoted PT(s), and terminating states are the parents of sf.
  • Markovian flows: A Markovian flow can be sampled by repeatedly drawing forward transitions from s0 until sf, terminating at state s with probability PT(s).The corresponding forward transition probabilities factorize trajectory probabilities and recover the terminating-state distribution.
  • Markovian flows: Each Markovian flow is uniquely specified by total flow and either forward probabilities, backward probabilities, or terminating flows plus non-terminating backward probabilities.These are three alternative parameterizations of the same Markovian flow.
  • Flow matching conditions: State and edge assignments define a flow exactly when they satisfy flow-matching conditions, while forward and backward probabilities jointly define one when detailed balance holds.For each edge, detailed balance requires ˆF(s) ˆPF(s′ | s) = ˆF(s′) ˆPB(s | s′), yielding a unique compatible Markovian flow.
  • Flow equivalence: Any flow has a unique equivalent Markovian flow, and equivalent Markovian flows are equal.Thus, learning can focus on Markovian flows because edge-flow specifications are generally exponentially smaller than trajectory-flow specifications, subject to flow-matching conditions.

3 GFlowNets: Learning a Flow

This section formalizes GFlowNets as estimators of Markovian flows and develops several valid ways to parametrize those flows. Configurations that encode valid flows induce trajectory distributions whose terminating-state probabilities are proportional to the target reward.

  • Learning a Flow: GFlowNets estimate Markovian flow functions or transition probabilities and sample complete trajectories by iteratively applying approximate forward transitions.The trajectory starts at s0 and continues until the sink state sf.
  • GFlowNets as an Alternative to MCMC Sampling: MCMC may require potentially very long chains and suffer mode-mixing, whereas GFlowNets amortize computation during training and construct each new configuration in a single generative pass.This shifts computational demand from repeated chain simulation to training the generator.
  • Abstract Flow Parametrization: A flow parametrization consists of a configuration space O, a map Π from configurations to trajectory distributions, and an injective map H from Markovian flows to configurations.The parametrization must preserve the probability measure associated with each represented flow.
  • Examples of Flow Parametrizations: Edge flows, forward transition probabilities, and forward-plus-backward transition probabilities each provide valid flow parametrizations.For each construction, the induced trajectory measure equals the probability measure associated with the corresponding Markovian flow.
  • GFlowNet Definition: For any represented Markovian flow, the induced terminating-state probability measure is proportional to the target reward R.This property follows from the definition of a GFlowNet and the associated flow theorems.
  • Offline Training: Training only on trajectories leading to observed high-reward terminal states is insufficient because the empirical trajectory distribution may lack full support.Combining those trajectories with a training distribution covering all possible trajectories guarantees recovery of a flow-matching model under the stated offline property.

4 Conditional Flows and Free energies

Conditional and state-conditional GFlowNets extend flow networks to indexed DAG families and enable marginalization over descendant terminal flows, supporting free-energy and related information estimates.

  • The initial flow F(s0) recovers the normalizing constant Z, which is also the partition function for the terminal reward function.
  • Ordinary state flow F(s) generally fails to marginalize rewards of all terminal descendants because upstream paths may bypass s.In the example, F(s2) = 4 while reachable terminating flows sum to 6.
  • 4 Free energies: Free energies formulate marginalization over many energy-function terms, opening applications in settings where expensive MCMC methods would otherwise be used.The paper applies this framework to joint distributions, including sets and graphs, and to marginal, conditional, entropy, and mutual-information estimates.
  • 4.2 Conditional flow networks: Conditional flow networks represent families of flow networks indexed by conditioning variables, including external rewards R(s | x) and previously visited states.Each indexed network inherits the corresponding flow, state-flow, edge-flow, and transition-probability concepts.
  • 4.3 State-conditional flow networks: A state-conditional flow network can be defined for any flow network and restores the desired marginalization property for terminal states s′ ≥ s.Its initial flow corresponds to the free energy associated with the descendant terminal-flow energy function.
  • 4.5–4.7 Extensions: Two GFlowNets—one for a target reward and one for its entropic reward—estimate entropy; conditional versions extend this to conditional entropy and mutual information.The paper also describes reward-conditional flows parameterized by θ for learning an Energy-Based Model.

5 GFlowNets on Sets, Graphs, and to Marginalize Joint Distributions

GFlowNets on sets and graphs represent compositional distributions and support probabilities, marginalization, conditioning, partition functions, and entropy-related quantities. Their set-based construction extends to joint-variable models, graphs, and modular energy-based structures.

  • Set GFlowNets: Set GFlowNets construct subsets by adding elements from a universe, with transitions constrained to legal set-building actions.The set state space contains all subsets plus a terminal state, and each nonterminal action adds one previously absent element.
  • Set GFlowNets: The framework directly supports marginal probabilities over all supersets of a given set, while estimated probabilities need not exactly normalize when learned flows are approximate.The text notes that estimated state and conditional probabilities are not guaranteed to sum to 1.
  • Set GFlowNets: A trained set GFlowNet can represent and sample a set distribution, estimate set probabilities and partition functions, and search for low-energy sets.It also supports conditional sampling over supersets, marginal probabilities of subsets, and entropy estimates.
  • GFlowNets on Graphs: Graphs are handled as sets of nodes and edges, with allowable actions restricted to enforce properties such as connectivity or acyclicity.This framework has been used to approximate posterior distributions over Bayesian-network graphical structures.
  • Marginalizing Joint Distributions: GFlowNets can estimate marginal probabilities over unspecified variables by treating variable-value pairs as set elements and continuing generation from a partially specified state.The construction allows variables to be sampled in any order permitted by the backward policy and supports completion of missing variables.
  • Modular Energy Function Decomposition: Graph GFlowNets can represent factor graphs whose reusable energy terms and generation modules correspond to distinct mechanisms.Given partial observations, they can sample latent factor graphs connecting observed and latent variables and use marginalization to compute free energies.

6 Continuous or Hybrid Actions and States

GFlowNets can be extended to continuous or hybrid states and actions by replacing discrete sums with integrals and decomposing transitions into discrete and continuous choices. The main challenge is representing and sampling tractable continuous output densities while preserving flow constraints.

  • Continuous and Hybrid Extensions: Continuous or hybrid GFlowNets replace sums over states or actions with integrals where variables include continuous components.The extension applies to spaces containing both discrete and continuous components.
  • Hybrid Transitions: Hybrid transitions can be decomposed into a discrete next-state choice followed by a continuous choice conditioned on that discrete choice.This yields a joint flow formulation with forward and backward conditional densities that can be computed exactly and made compatible.
  • Continuous Outputs: The central practical challenge is representing continuous output densities so their values can be evaluated and samples generated.Categorical transitions are standard, whereas continuous conditionals require explicit density parameterizations.
  • Continuous Outputs: Gaussian conditionals provide tractable normalization but may restrict capacity and hinder minimization of detailed-balance or flow-matching losses.Mixture models can increase flexibility by partitioning continuous densities into components.
  • Flow Parameterizations: Edge-flow parameterizations and lower-level GFlowNet–energy-function pairs offer alternatives for exactly satisfying detailed balance or modeling complex transition families.The lower-level construction can sample transitions in either direction and evaluate their normalizing constants.
  • Related Work: Related work extended GFlowNet theory to general state spaces, including continuous ones, and experimentally validated usual losses for inference in continuous domains.This passage attributes the extension and validation to Lahlou et al. (2023).

7 Related Work

GFlowNets differ from established generative, reinforcement-learning, and MCMC approaches in their reward-based training objective and emphasis on sampling diverse outcomes. Their amortized sampler exchanges long-chain sampling costs for training costs, with benefits depending on generalization and structure in the target distribution.

  • GFlowNets are trained from energy or reward functions, unlike VAEs and GANs, which are trained on finite samples from the target distribution.
  • Their reward signal distinguishes likely, unlikely, and intermediate configurations rather than labeling only observed examples as positive.
  • 7.2 Contrast with Regularized Reinforcement Learning: GFlowNets sample terminating states in proportion to reward, whereas MaxEnt RL can overweight states by the number of trajectories leading to them.The methods coincide only when the trajectory DAG is a tree rooted at the initial state.
  • 7.3 Contrast with Monte-Carlo Markov Chain methods: MCMC mode mixing can require exponentially long chains when modes are separated by long sequences of low-probability configurations.Randomized chains and simulated annealing become less effective in high dimensions when modes occupy tiny volumes.
  • 7.3 Contrast with Monte-Carlo Markov Chain methods: Amortized GFlowNet sampling replaces the complexity of long MCMC chains with the complexity of training a sampler.This can help when the target distribution has generalizable structure that supports jumping from known modes to distant, unvisited modes.
  • 7.3 Contrast with Monte-Carlo Markov Chain methods: The effectiveness of amortized sampling depends on the learner’s ability to generalize through appropriate inductive biases; without structure, it need not outperform MCMC.Molecular graph experiments comparing GFlowNets and MCMC appear to support this dependence on structure.

8 Conclusions and Open Questions

The paper deepens GFlowNets’ mathematical foundations and broadens them to marginalization, alternative credit assignment, richer reinforcement-learning settings, and multiple learned flows. It identifies extensions and applications, while emphasizing that many formulations still require empirical validation and further development.

  • The paper introduces detailed balance loss, separating the backward policy governing construction order from constraints imposed by the target reward.
  • Conditional GFlowNets generalize partition-function estimation to intractable sums of rewards over terminal states reachable from arbitrary states.This provides a framework for marginalization and free-energy estimation.
  • Direct credit-assignment objectives are proposed as alternatives to flow matching that may bypass slow propagation of credit from sequence ends to beginnings.
  • The framework is extended to intermediate rewards and stochastic environments, clarifying links with reinforcement learning and supporting return-based sampling.
  • Learning multiple flows simultaneously opens distributional and unsupervised GFlowNets and sampling from Pareto fronts defined by multiple reward functions.
  • Open questions include continuous actions and states, hierarchical GFlowNets, integrated energy parametrizations, and empirical validation of the mathematical formulations.

Appendix A. Direct Credit Assignment in GFlowNets

The section develops direct credit-assignment estimators for GFlowNets by propagating flow perturbations through matched-flow constraints. These estimators can provide long-range training signals, but their unbiasedness is asymptotic and depends on how trajectories are sampled.

  • Motivation: Flow-matching losses can require many updates and trajectories to propagate terminal reward mismatches backward through long trajectories.This motivates alternative objectives that assign credit more directly across the trajectory.
  • Flow perturbations: Under matched flows, perturbing an upstream edge changes downstream log-flow according to a backward conditional probability over trajectories.The corresponding forward relation uses conditional trajectory probabilities in the opposite direction.
  • Estimators: The proposed total-derivative estimator is asymptotically unbiased when flows are matched, including convex combinations of forward- and backward-transition estimators.The endpoint cases λ = 1 and λ = 0 are each asymptotically unbiased, so their convex combination remains unbiased.
  • Interpretation: The resulting training signal resembles policy gradient by assigning information to earlier states, while relying on the flow-matching fixed point.The connection is exact only under the stated matched-flow conditions.
  • Limitations: With off-policy or replay-buffer trajectories, importance weighting can correct distribution mismatch but may introduce variance, while slight mismatch can still induce bias.The estimator is unbiased for trajectories sampled from the current GFlowNet distribution; practical flow mismatch also contributes bias.

Appendix B. Conditional GFlowNets

Conditional GFlowNets jointly represent and train flow configurations across conditioning variables, enabling simultaneous sampling from condition-specific reward functions. Their losses can decompose by condition and state, while shared parameterizations trade capacity across conditions.

  • Definition and semantics: A conditional GFlowNet associates each condition x with a DAG, reward function, flow configuration, and induced trajectory distribution.For a flow-matched configuration, the terminal distribution under condition x is proportional to R_x(s).
  • Training objectives: Conditional flow-matching losses can be condition-decomposable, and particular constructions are also state-, edge-, or trajectory-decomposable.The conditional loss is zero exactly when every condition admits a compatible flow-matched configuration.
  • Amortized parameterization: Shared parameterization lets a model learn flows as functions of both conditions and non-terminating edges, exploiting generalization across conditions.This replaces learning each condition-specific flow independently with a common function-space representation.
  • Limitations: Finite model capacity can make zero loss unattainable, leaving the optimized shared model to trade off performance across conditions.The theoretical zero-loss guarantee remains valid, but optimization is coupled through shared parameters.
  • Environment decomposition: The framework extends the action-state decomposition so policies select actions and the environment maps action-state pairs to subsequent states.Backward detailed balance must separately invert the policy step and represent the environment-dependent step.

C.2 Unknown Deterministic Environments

For unknown deterministic environments, GFlowNets must learn both the transition function and its inverse. This can be difficult when discrete high-dimensional spaces require generalization to unseen transitions.

  • Transition learning: An unknown deterministic environment requires learning T and an inverse T^-1 that recovers the previous state from the next state and action.The inverse satisfies T^-1(T(s, a), a) = s.
  • Scope boundary: Learning these forward and inverse transitions may be difficult in high-dimensional discrete spaces when generalization to unseen transitions is required.The paper notes that observed transitions may cover only an exponentially tiny fraction of future combinatorial transitions.

C.3 Stochastic Environments

Stochastic environments preserve the possibility of constructing Markovian flows, but environment-controlled transitions restrict backward probabilities and can make target terminal rewards unattainable. The section formalizes this limitation and illustrates it with a counterexample.

  • Core result: In stochastic environments, any policy can yield a Markovian flow, but the desired terminal reward flows may not be perfectly achievable.Flow matching or detailed balance can still define the flow, even when the environment prevents the target terminal distribution.
  • Practical considerations: With sufficient capacity and training time, the construction yields a Markovian flow, although finite capacity and training time can prevent exact target matching even in controllable environments.Unknown environment transitions can be estimated from observed state-action-next-state triplets.
  • Unattainable rewards: A terminal state with positive reward but zero environment probability cannot receive the desired flow under any policy.This establishes unattainability directly from the environment’s support constraint.
  • Counterexample: Figure 6 shows that a zero-probability environment transition constrains the corresponding backward probability to zero, preventing free backward-transition selection while preserving flows and rewards.The example has two paths to the same terminal state, but one path is ruled out by the stochastic environment.
  • Constraint: The forward transition on odd-to-even edges is fixed by the environment, so the corresponding backward transition cannot generally be chosen freely.Even-to-odd backward transitions are fixed by construction, whereas odd-to-even transitions inherit the environment’s probabilities.

Appendix D. Expected Downstream Reward and Reward-Maximizing Policy

The paper defines expected reward under terminating-state distributions and shows that greedy policy improvement cannot reduce it. For the GFlowNet distribution proportional to reward, related flows and additional heads provide ways to estimate expected rewards.

  • For the GFlowNet terminal distribution PT, the expected reward has a direct expression derived from the conditional terminating-state probabilities.This expression is established by applying the definition of conditional PT.
  • Expected reward is defined for any terminating-state distribution induced by a policy, paralleling state and state-action value functions in reinforcement learning.The paper uses this expected reward to formulate a policy-improvement result.
  • A greedy policy under the expected reward achieves an expected reward no worse than that of the original policy.The result follows by selecting, at each state, the action with the highest expected reward.
  • When PT(s) = R(s)/Z, the greedy policy associated with the GFlowNet policy maximizes expected reward for all states.This is stated as an immediate corollary of the reward-maximizing policy result.
  • A second flow trained with reward R2 can estimate the expected reward under the distribution generated by a flow trained on R.The proposition identifies the numerator with the self-flow of the second flow and the denominator with the self-flow of the first.

D.1 Preference for High-Reward Early Trajectory

The backward policy can be chosen to favor construction orders whose state-action pairs have larger expected downstream reward.

  • A backward policy preference for larger expected reward can make GFlowNet sampling favor high-reward states earlier in construction.The backward transition probabilities are defined to assign more mass to state-action pairs with larger expected value.

Appendix E. Intermediate Rewards and Trajectory Returns

The paper extends GFlowNets from terminal rewards to accumulated intermediate rewards by augmenting states with trajectory returns. A trained return-augmented GFlowNet then samples accumulated rewards in proportion to their values, while requiring offline training.

  • The expected future return from a state satisfies a recursion analogous to the undiscounted episodic Bellman equation.The paper contrasts value propagation through forward transitions with flow propagation through backward transitions.
  • Intermediate environment rewards can be accumulated along partial trajectories and incorporated into the GFlowNet state.A return-augmented state includes the accumulated reward ν(s), defining a return-augmented GFlowNet.
  • A fully trained return-augmented GFlowNet samples accumulated reward ν with probability proportional to ν.This follows by setting the terminal reward function equal to the accumulated environment reward.
  • Training such a GFlowNet is restricted to offline trajectories whose returns have already been observed.The limitation arises because the training procedure requires terminated trajectories and their observed returns.
  • Return-augmented GFlowNets can handle stochastic environment rewards because the accumulated reward is included in the state.Without state augmentation, the terminal reward would not be a function of the state.

Appendix F. Multi-Flows, Distributional GFlowNets, Unsupervised GFlowNets and Pareto GFlowNets

The paper develops outcome-conditioned and distributional GFlowNets for conditioning generation on specified outcomes or returns. These methods support selective sampling but rely on offline training and can become difficult to train when outcomes are numerous.

  • An outcome-conditioned GFlowNet takes a known state-derived outcome y = f(s) as input and models trajectories consistent with that outcome.The outcome may be a reward value or a feature vector sufficient to determine reward functions.
  • When trained to completion, an outcome-conditioned GFlowNet samples only terminating states that yield the chosen outcome.The paper focuses on discrete outcomes while anticipating generalization to continuous outcomes.
  • A distributional GFlowNet conditions on the environment return of complete trajectories and can obtain that return from a return-augmented terminating state.This combines outcome conditioning with return augmentation.
  • Outcome-conditioned training is offline because conditioning information such as final return may become known only after trajectory sampling.The proposed contrastive procedure begins with an unconditional training-policy trajectory and then updates using its observed outcome.
  • If many outcomes are possible, conditional matching becomes unlikely or sparse because a sampled trajectory may not reproduce the conditioning outcome.The procedure samples a conditional trajectory, observes its actual outcome, and uses an indicator reward for matching outcomes.
  • A smoother conditional reward than the sharp indicator R(s|y) = 1f(s)=y is proposed as future work to ease training.The motivation is to replace a sparse reward with a less discontinuous signal.

F.1 Defining a reward function a posteriori

An outcome-conditioned GFlowNet can be converted after training into one that samples according to a posteriori specified rewards based on outcomes. This flexibility requires additional computation at run time.

  • F.1 Defining a reward function a posteriori: The reward function may be chosen after training, including when it is initially unknown or known only up to unspecified constants.This supports posterior reward specification and generalization to more unsupervised training.
  • F.1 Defining a reward function a posteriori: Proposition 62 converts an outcome-conditioned GFlowNet into one matching any terminal reward R = r ◦ f specified through the outcome y = f(s).The conversion uses the outcome-conditioned flow F(A|y) to construct the reward-specific flow F_r◦f(A).
  • F.1 Defining a reward function a posteriori: The resulting policy π_r◦f(a|s) can be obtained from the outcome-conditioned state flow, state-action flow, and action policy.These quantities are denoted F(s|y), F((s,a)|y), and π(a|s,y), respectively.
  • F.1 Defining a reward function a posteriori: The construction is reward-independent at the trajectory-conditioning level because backward-path probabilities do not depend on the reward function.This makes the flow identity valid for any function r.
  • F.1 Defining a reward function a posteriori: An indicator reward at a specific outcome reduces the construction to the corresponding outcome-conditioned flow F(A|y).The proof then recovers the general result by substituting the conditional-flow definition, exchanging sums, and applying the indicator.
  • F.1 Defining a reward function a posteriori: At run time, reward-specific predictions and sampling require additional computation, potentially including sums or Monte Carlo integration over outcomes.The outcome-conditioned network can be converted on the fly without retraining.

F.2 Pareto GFlowNets

Pareto GFlowNets condition on objective trade-off parameters to represent families of reward functions. Scanning these parameters yields policies, probabilities, or free energies for different trade-offs and supports Pareto-frontier sampling.

  • F.2 Pareto GFlowNets: Pareto additive and multiplicative terminal rewards combine multiple objectives using convex weights over a discrete set of weight choices.The objectives are represented as y_i = f_i(s), with weights satisfying the convex-combination constraint.
  • F.2 Pareto GFlowNets: A conditional GFlowNet can use the weight vector ω as input and R_ω(s) = R(s|ω) as its conditional terminal reward.This trains one model for the family of reward functions indexed by ω.
  • F.2 Pareto GFlowNets: Scanning the available ω values produces different policies, predicted probabilities, or free energies.The same parameterization can extend to non-convex and non-linear objective combinations when indexed by ω.
  • F.2 Pareto GFlowNets: After training, sampling ω first and then trajectories draws solutions from different trade-off points on the Pareto frontier.This is useful for obtaining diverse solutions in multi-objective optimization or sampling.
  • F.2 Pareto GFlowNets: Objective vectors y can also condition an outcome-driven GFlowNet, with prior structure represented through a basis expansion of the flow.The proposed basis representation can exploit assumptions that objectives are independently modeled.
Loading 2111.09266v5…