PaperlayerView exact revision on arXiv

Source-linked AI summary

Your Probabilistic JEPA Is Secretly a Hidden Markov Model: A State-Space Interpretation of Joint-Embedding Predictive Learning

Yongchao Huang

arXiv:2608.13621v1cs.AI

TL;DR

The paper asks when a fully stochastic temporal JEPA can be understood as an HMM rather than merely resembling one. It formalizes progressively stronger correspondences, introduces MCJEPA and broader transition constructions, and reports controlled evidence for composition, filtering, predictive Markovization, and objective-level distinctions. The resulting view is an HMM-compatible latent state-space interpretation whose strength depends on probabilistic components and training objectives.

  • Problem

    The paper addresses the gap between JEPA’s latent prediction architecture and an HMM’s sequence-level probabilistic semantics, including whether their computational roles imply stronger equivalence.

  • Method

    The paper formalizes four correspondence levels, introduces MCJEPA with matrix-based transitions, generalizes to other Markov dynamics, and analyzes explicit, inverse, and implicit emissions.

  • Results

    Controlled experiments support transition composition, the filtering interpretation, predictive Markovization, and the distinction between JEPA latent prediction and HMM-style sequence learning.

  • Takeaways & Limitations

    Temporal JEPA can provide an HMM-compatible latent state-space architecture, but the degree of HMM equivalence depends on its probabilistic components and training objective.

  • Takeaways & Limitations

    An implicit emission establishes only a static observation–state correspondence unless state evolution and context encoding also satisfy sequence-level consistency conditions.

Abstract

from arXiv · show

A hidden Markov model (HMM) combines three roles: inference of a hidden-state belief from observations, propagation through a Markov transition, and emission back to observation space. We show that full, time-indexed Predictive Information Bottleneck VJEPA (PIB-VJEPA) exposes the same computational structure: a stochastic context encoder plays the role of an amortized filtering distribution, a probabilistic predictor defines latent-state dynamics, and a decoder, inverse target encoder, or induced implicit conditional supplies the emission direction. We distinguish 4 progressively stronger levels of correspondence and give sufficient conditions for exact sequence-level HMM equivalence. To make the connection concrete, we introduce Markov-Chain JEPA (MCJEPA), which replaces the latent predictor by a learned transition matrix; in the finite time-homogeneous case, matrix powers guarantee exact multi-horizon Chapman--Kolmogorov consistency. Conditioned discrete-state transitions, continuous-state Markov kernels, and continuous-time dynamics extend this construction, while deterministic temporal JEPA appears as a degenerate Dirac-kernel special case. We further interpret predictive information-bottleneck learning as seeking a compact predictive state: compression promotes minimality, while residual predictability tests sufficiency. Controlled experiments support transition composition, the filtering interpretation, predictive Markovization in a known synthetic process, and the distinction between JEPA latent prediction and HMM-style sequence learning. Together, these results give temporal JEPA a principled state-space interpretation.

1 Introduction

The paper argues that full, time-indexed PIB-VJEPA exposes an HMM-like encode–transition–emit structure, while emphasizing that architectural correspondence alone does not guarantee sequence-level or objective equivalence. It develops this claim through progressively stronger correspondences, MCJEPA, generalized transitions, and controlled experiments.

  • Probabilistic formulation: The full PIB-VJEPA formulation makes current representations, future targets, and latent transitions probabilistic, providing the clearest setting for the state-space interpretation.The online encoder infers the current predictive-state distribution, the transition propagates it, and the target encoder supplies the future latent distribution used for training.
  • Core analogy: Full PIB-VJEPA maps stochastic context encoding to hidden-state inference, probabilistic prediction to latent transition, and decoder or inverse target encoding to emission.The encoder runs from observations to states; the optional decoder or inverse target encoder runs from states back toward observations.
  • Levels of correspondence: The paper distinguishes computational correspondence, emission-complete representation, sequence-level HMM equivalence, and model-and-objective equivalence.Exact sequence-level equivalence additionally requires Markov, emission, marginal-consistency, and filtering-consistency conditions.
  • MCJEPA and extensions: MCJEPA replaces the latent predictor with a learned transition matrix whose powers guarantee exact direct-versus-composed multi-step consistency.The construction is extended to conditioned discrete transitions, continuous-state Markov kernels, and continuous-time dynamics; deterministic temporal JEPA is the degenerate Dirac-kernel case.
  • Experiments: Four controlled experiments examine transition recovery and path consistency, filtering, predictive compression and Markovization, and the distinction between JEPA and HMM-style sequence learning.Together, these experiments test progressively stronger aspects of the proposed HMM interpretation.

2 From JEPA to Markov-Chain JEPA

The paper organizes latent Markov dynamics across discrete or continuous time and state spaces, then instantiates the discrete-time, discrete-state case as MCJEPA. MCJEPA aligns stochastic encoding, matrix-based latent propagation, target matching, and optional observation-space mapping, with matrix powers ensuring exact multi-horizon consistency.

  • 2.1 Latent Markov dynamics: Latent Markov dynamics span four cases formed by crossing discrete or continuous time with discrete or continuous latent state spaces.The main construction uses discrete time and discrete state; broader variants use conditioned matrices, Markov kernels, generators, or stochastic differential equations.
  • 2.2 The HMM–PIB-VJEPA analogy at a glance: PIB-VJEPA follows an encode–transition–emit pipeline: the context encoder infers a latent belief, the predictor advances it, and a decoder or inverse target encoder can map it to observations.The context encoder matches the HMM inference role exactly only under later filtering-consistency conditions; the observation model is optional for the core JEPA objective.
  • 2.4 MCJEPA: replace the predictor by a Markov chain: MCJEPA uses a categorical predictive state, a stochastic online encoder, an EMA target encoder, and a fixed time-homogeneous row-stochastic transition matrix.The matrix represents categorical transition probabilities, while the target distribution supplies the future latent training target.
  • 2.4 MCJEPA: replace the predictor by a Markov chain: Training matches each predicted distribution to the corresponding target-encoder distribution across selected horizons, using weights that may be uniform or exponentially discounted.Discounted weights emphasize near-term predictions, while γ = 1 recovers uniform weighting.
  • 2.5 Exact multi-horizon path consistency: Matrix powers produce exact path consistency: direct prediction over h1 + h2 steps equals composition through the intermediate horizon.Under finite, time-homogeneous Markov dynamics with fixed transition matrix A, all decompositions of the same total horizon yield the same predictive distribution.

3 From a Transition Matrix to a Neural Markov Kernel

The fixed MCJEPA transition matrix generalizes to side-information-conditioned matrices, continuous-state Markov kernels, and continuous-time dynamics. Recurrent predictors become first-order Markov after augmenting the state with memory, while deterministic JEPA is recovered as a Dirac-kernel limit.

  • 3.1 Conditioned discrete-state transitions: The Markov principle requires next-state predictions to depend on history only through the current predictive state and transition-relevant side information.This permits fixed or conditioned transitions and discrete or continuous latent states.
  • 3.1 Conditioned discrete-state transitions: Conditioned transition matrices support actions, goals, regimes, covariates, and irregular intervals while preserving exact composition for identical chronological transition sequences.Different transition matrices need not commute, so chronological ordering matters.
  • 3.2 Continuous-state stochastic transitions: Continuous-state models replace matrices with Markov kernels that distribute probability over the next latent state, with multi-step prediction obtained by marginalizing intermediate states.Composition may require sampling, Monte Carlo integration, moment propagation, or learned approximations unless the transition family is restricted.
  • 3.2 Continuous-state stochastic transitions: A single Gaussian kernel cannot represent genuinely multimodal conditionals, so mixtures, flows, or diffusion-based transitions may be needed while retaining Markov factorization.The central requirement is the Markov factorization, not Gaussianity.
  • 3.3 Continuous-time latent dynamics: Continuous-time extensions use generator matrices for discrete states or stochastic differential equations for continuous states, with diffusion zero yielding deterministic dynamics.A fixed generator propagates across intervals through a matrix exponential; varying conditioning within an interval requires time-varying generator composition.
  • 3.5 When a recurrent predictor is Markov: A recurrent predictor is not first-order Markov in Z_t alone when identical latent states retain different memories, but augmenting the state with memory restores first-order representation.PIB-VJEPA therefore seeks a predictive state that summarizes history relevant to future prediction; otherwise memory or higher-order dynamics may be required.

4 A Probabilistic JEPA Is Secretly an HMM

PIB-VJEPA mirrors HMM computation through stochastic state inference, latent transition, and an observation-space emission direction. Exact sequence-level equivalence requires additional consistency conditions beyond sharing these components.

  • Architecture: PIB-VJEPA uses an online encoder for current latent states, a probabilistic predictor for future states, and a target encoder for training targets.The target encoder is slowly updated and supplies the future latent distribution against which prediction is trained.
  • Architecture: The PIB-VJEPA objective combines future-state prediction with source- and target-side regularization that controls retained information and latent regularity.The coefficients trade predictive accuracy against compression and regularity.
  • Emission constructions: The stochastic encoder is a state-inference distribution, not an HMM emission, because it maps observations to latent states rather than latent states to observations.Only an inverse target encoder or decoder supplies the state-to-observation direction.
  • Emission constructions: An HMM emission direction can come from an explicit probabilistic decoder, an invertible target encoder, or an implicit conditional induced by a stochastic encoder and data marginal.The implicit construction is normalized and makes the encoder the exact posterior in a static latent-variable model.
  • Levels of correspondence: Exact sequence-level HMM equivalence requires Markov latent dynamics, a valid state-to-observation conditional, transition-consistent marginals, and an encoder equal to the Bayesian filtering posterior.The implicit emission construction alone establishes only a static observation–state correspondence.
  • Levels of correspondence: The correspondence hierarchy separates shared computational roles, emission-complete representations, sequence-level probabilistic equivalence, and HMM-style sequence training.A shared architecture does not by itself imply a common joint distribution or objective.

5 Information Bottleneck Learning as Markovization

Predictive information-bottleneck learning seeks states that compress observation history while retaining information useful for future prediction. Under predictive sufficiency, the learned state screens off older history from the next latent state.

  • Objective: The predictive information bottleneck balances compression of observation history against predictive dependence between current and future latent states.The practical PIB-VJEPA objective implements this principle through latent prediction and variational bottleneck regularization.
  • Predictive sufficiency: Predictive sufficiency requires that the current state retain all history information relevant to the future.Once the predictive state is known, remaining observation history should provide no additional information about future outcomes.
  • Limitations: Minimizing the bottleneck objective alone does not guarantee predictive sufficiency of the learned representation.The sufficiency condition is stronger than the optimization objective itself.
  • Predictive sufficiency: For one-step prediction, predictive sufficiency implies that older observation history provides no additional information about the next latent state after conditioning on the current state.With transition-relevant side information, the diagnostic also conditions on that side information.

6 Residual Predictability as a Diagnostic of Markov Sufficiency

Residual-predictability diagnostics test whether older history improves prediction beyond the current latent state and transition inputs. Such gains indicate a failure of the relevant conditional-mean sufficiency, but negative findings do not establish full Markov sufficiency.

  • Diagnostic limits: The covariance diagnostic detects only linear dependence, so a near-zero value does not establish Markov sufficiency.It rules out only the tested form of linear residual dependence.
  • Categorical diagnostics: A conditionally mean-correct transition has zero expected residual given its current state and available transition inputs.This property underlies the categorical probability-innovation diagnostic.
  • History-augmentation diagnostics: A positive history-augmentation gain means older latent history improves prediction beyond the current state and side information.This provides evidence against conditional-mean sufficiency of the current state–predictor pair.
  • Interpretation: Residual predictability can reflect either an insufficient representation or a restricted or poorly optimized transition model.Separating these effects requires distinguishing current-input prediction from additional prediction using older history.
  • Diagnostic limits: History-augmentation evidence requires matched capacity, regularization, and out-of-sample evaluation to avoid attributing model-flexibility differences to representation insufficiency.Unequal diagnostic flexibility can create an apparent history gain without genuinely additional predictive information.
  • Diagnostic limits: Zero autocorrelation and calibrated marginal diagnostics do not establish the nonlinear conditional independence required for Markov sufficiency.Different diagnostics test only particular consequences of correct conditional prediction.

7 Experiments

The experiments test whether probabilistic temporal JEPA can recover coherent Markov structure, filter ambiguous observations, and learn compact predictive states. They also show that HMM-style supervision improves probabilistic recovery, while JEPA-only training preserves latent prediction without directly optimizing sequence likelihood.

  • Experiment 1: finite-HMM recovery and Markov composition: MCJEPA enforces exact multi-horizon path consistency, whereas independently learned horizon predictors can fit more flexibly but define a less faithful transition law.The shared transition matrix makes Dpath identically zero; independently parameterized Ah need not satisfy composition identities and may achieve slightly better predictive NLL under ambiguity.
  • Experiment 1: finite-HMM recovery and Markov composition: Occupancy and entropy regularization address complementary collapse modes, and using both yields confident, diverse state assignments without collapse.Occupancy encourages global state use, while entropy encourages confident per-example assignments; entropy alone causes four of five runs to use one effective state.
  • Experiment 2: filtering resolves emission ambiguity: Filtering combines current observations with propagated state beliefs, remaining stable through locally ambiguous measurements and changing when accumulated evidence supports a transition.This distinguishes the filtering belief p(St | X≤t) from local observation evidence p(St | Xt).
  • Experiment 3: predictive compression and Markovization: Predictive compression removes redundant history without sacrificing prediction when it retains the known four-state sufficient representation, but further compression becomes insufficient.The four-state representation preserves the same predictive information and prediction NLL as the full history, whereas Zt = Xt reduces predictive information from 0.2807 to 0.0192 and raises NLL from 0.3978 to 0.6593.
  • Experiment 3: predictive compression and Markovization: Residual-history testing distinguishes sufficiency from minimality: previous-state information helps when compression omits transition-relevant history, but not when the current representation is already sufficient.The four-state representation removes redundant history, while the two-state representation leaves substantial predictive information in the previous representation.
  • Experiment 4: HMM-style training of PIB-VJEPA: HMM-style sequence and filtering supervision improves probabilistic latent-state recovery, while JEPA-only training learns predictive latents without directly maximizing observation-sequence likelihood.The hybrid improves over JEPA-only training on every reported metric for every seed and nearly matches HMM-style sequence likelihood; the HMM sequence-plus-filter-distillation regime omits LMC yet achieves the strongest state recovery and sequence likelihood.

8 Discussion

The paper frames probabilistic temporal JEPA as an HMM-like state-space system, while emphasizing that architectural correspondence does not automatically establish sequence-level or objective-level equivalence. This interpretation treats representation learning as the search for a compact, sufficiently predictive Markov state.

  • HMM correspondence: PIB-VJEPA separates latent-state inference, Markov propagation, and state-to-observation emission, but these roles alone do not establish HMM sequence equivalence.The paper distinguishes computational correspondence, emission-complete representation, sequence-level equivalence, and model-and-objective equivalence.
  • Markov dynamics: A nonlinear neural predictor remains first-order Markov whenever its conditional distribution depends only on the current latent state and available side information.Markovianity is defined by conditional independence, not by restricting the predictor’s functional form.
  • Markov dynamics: If prediction depends on memory or multiple prior states, augmenting the latent state can restore a first-order interpretation when it contains transition-relevant history.Without such augmentation, exact HMM correspondence using the unaugmented state does not follow.
  • Predictive state: A first-order predictor does not necessarily produce a sufficient Markov state, because older history may remain predictive after conditioning on the current representation.Residual-history predictability diagnoses information omitted from the representation.
  • Predictive state: Predictive information bottleneck learning balances compression, which promotes minimality, with residual predictability, which tests sufficiency.Together, these criteria can support approximate Markovization rather than merely imposing a first-order predictor.
  • Training semantics: Architecture and objective are separate: JEPA latent alignment need not maximize observation-sequence likelihood, whereas HMM-style supervision improves recovery of probabilistic latent-state structure.The paper reports a hybrid objective approaching HMM-level sequence likelihood and transition recovery while retaining EMA-target prediction.
  • Training semantics: Observation reconstruction remains optional for predictive representation learning, but decoders become necessary when observation forecasting or sequence likelihood is required.An invertible target encoder is another observation-space construction, though exact invertibility can conflict with bottleneck compression.

9 Conclusion

The conclusion makes the state-space interpretation concrete through MCJEPA and specifies when temporal JEPA becomes an HMM. It also extends the view beyond finite chains and interprets bottleneck learning as compact predictive-state construction.

  • Conclusion: MCJEPA replaces the latent predictor with a learned row-stochastic transition matrix, whose powers provide exact multi-horizon Chapman–Kolmogorov consistency.Direct and composed predictions agree because all horizons use powers of the same matrix.
  • Conclusion: The construction extends to conditioned discrete transitions, continuous-state Markov kernels, and continuous-time dynamics, with deterministic JEPA as a degenerate Dirac-kernel case.The deterministic boundary is represented by point-mass latent distributions and Dirac transitions.
  • Conclusion: Temporal JEPA supplies HMM-like inference, latent transition, and emission roles through its context encoder, probabilistic predictor, and decoder, inverse encoder, or implicit conditional.The paper distinguishes these structural roles from stronger sequence-level and objective-level equivalence.
  • Conclusion: Exact sequence-level HMM representation requires sufficient transition, marginal-consistency, and filtering-consistency conditions.The stronger model-and-objective equivalence additionally requires HMM-compatible sequence-level probabilistic training.
  • Conclusion: Predictive information bottleneck learning seeks a compact predictive state by removing redundant history while testing whether transition-relevant information remains outside the representation.Compression promotes minimality, whereas residual predictability tests sufficiency.

A Training Objectives and Minimal Algorithm

The training procedure uses a shared transition matrix for multi-horizon latent prediction, making path consistency automatic. Horizon-dependent matrices require an additional consistency penalty, while residual-history diagnostics are evaluated after fitting.

  • Shared transition: In shared-A MCJEPA, every h-step prediction uses a power of the same transition matrix, so exact Chapman–Kolmogorov consistency requires no extra penalty.This follows from associativity of matrix multiplication and holds for every decomposition of a total horizon.
  • Shared transition: A horizon-dependent transition-matrix variant loses guaranteed path consistency and may require a weighted Chapman–Kolmogorov penalty.The penalty is unnecessary for the shared-A model used in the main experiments.
  • Training algorithm: A minimal training step samples a time, horizon, history, target, and side information before computing the current-state distribution.The procedure then obtains the EMA target, propagates through the shared matrix, evaluates latent prediction and state-use losses, and updates online and target parameters.
  • Training algorithm: Conditioned discrete-state training replaces shared-matrix propagation with ordered transition composition, leaving the remainder of the training procedure unchanged.The conditioned model uses the corresponding transition composition defined for that setting.
  • Diagnostics: Residual-history gain is evaluated after representation and transition fitting as a held-out diagnostic rather than included in the default MCJEPA objective.This separates testing omitted information from directly training the representation to satisfy the diagnostic.

B.2 Proof of Proposition 2

The appendix constructs and analyzes observation-space completions for latent JEPA models, then explains sequence likelihood and filtering objectives. Static implicit emissions are exact but do not by themselves yield a sequence-level HMM.

  • Implicit emission: An implicit conditional formed from a local stochastic encoder and the data marginal is normalized and reproduces the one-time observation–state joint distribution.This gives a valid static latent-variable model with observation marginal pdata(x).
  • Implicit emission: Static Bayes completion does not establish a sequence-level HMM, which additionally requires transition, marginal-consistency, and filtering consistency.A history-dependent encoder cannot automatically be reinterpreted as an emission model.
  • Emission constructions: An explicit decoder supplies the HMM-style state-to-observation direction and can function as a post-hoc probe or as part of joint observation-likelihood training.Finite observations and states can use an emission matrix, while continuous or high-dimensional observations require conditional densities.
  • Emission constructions: An invertible target encoder supplies a deterministic state-to-observation map without a separate decoder, but exact invertibility constrains architecture and can prevent unrestricted dimensionality reduction.A noise model is needed for a non-degenerate probabilistic emission.
  • Emission constructions: Implicit emissions may be intractable for sampling or likelihood evaluation because they depend on the generally unknown data marginal.The construction is exact statically but does not automatically provide a practical generative model.
  • Sequence objectives: HMM sequence likelihood trains transition and emission models, while filtering distillation trains the context encoder to approximate the Bayesian filtering belief.For categorical states, the forward recursion evaluates sequence evidence and the exact filtering posterior; the encoder can approximate that posterior in one pass.
  • Sequence objectives: For continuous or nonlinear latent states, exact marginalization is generally unavailable, so an approximate sequence posterior yields a variational lower bound.The posterior may be causal for online filtering or smoothing when full-sequence information is available.

D.4 Relation to JEPA and hybrid training

The paper separates JEPA latent prediction from HMM-style sequence learning, then identifies conditions under which temporal JEPA admits an exact HMM interpretation.

  • JEPA and hybrid training: JEPA latent prediction and HMM-style sequence learning remain distinct objectives, even when they share latent states and transitions.The hybrid regime combines JEPA prediction, HMM likelihood, and filtering distillation, while the transition matrix is shared across objectives.
  • JEPA and hybrid training: Adding an emission model alone does not make PIB-VJEPA training identical to HMM training.Equivalence also requires observation-sequence evidence and corresponding sequence-inference semantics to participate in learning.
  • HMM equivalence conditions: An arbitrary history encoder need not equal the Bayesian filter, so architectural correspondence is weaker than exact HMM equivalence.Independently induced one-time latent marginals may also fail to evolve under the learned transition kernel.
  • HMM equivalence conditions: Exact sequence-level equivalence requires Markov latent dynamics, a valid emission conditional, dynamically consistent latent marginals, and filtering-consistent context encoding.The emission may come from a decoder, an invertible target encoder, or an implicit local-evidence construction.
  • Implicit emission: The implicit-emission construction requires separating local observation evidence from the history-dependent filtering belief.The local stochastic encoder depends only on the current observation, whereas the context encoder represents the filtering distribution.

E.4 Completion of the proof for Theorem.1

The proof verifies that the four consistency conditions produce the conditional HMM factorization and make the context encoder its filtering distribution, while experiments remain small synthetic structural diagnostics.

  • Proof completion: The theorem’s four conditions establish first-order latent dynamics, a valid emission conditional, dynamically consistent marginals, and Bayesian filtering consistency.These conditions apply to explicit, inverse-target, or implicit emission constructions.
  • Proof completion: Under these conditions, the joint sequence distribution is precisely the HMM factorization and the context encoder represents its filtering distribution.This is the sufficient-condition conclusion of Theorem 1.
  • Experimental scope: The experiments use small synthetic settings to isolate structural claims under known latent dynamics rather than benchmark large-scale forecasting performance.Learned-model results use five seeds, while exact finite calculations are deterministic and formal hypothesis tests are not performed.
  • Evaluation: Evaluation separates representation recovery, transition recovery, predictive performance, probabilistic-model fit, path consistency, and filtering agreement.Observation-sequence NLL marginalizes the latent trajectory with the HMM forward algorithm, while filtering KL diagnoses amortized filtering alignment.
  • Evaluation: MCJEPA’s shared transition matrix guarantees zero direct-versus-composed path disagreement, unlike independently learned horizon-specific matrices.The guarantee holds algebraically up to numerical precision.

F.2 Experiment 1: finite-HMM recovery and Markov composition

Experiment 1 evaluates finite-HMM state recovery and multi-horizon prediction, showing strong recovery in separated emissions and exact composition for the shared-transition model.

  • Setup: The experiment generates observations from a four-state stationary Gaussian HMM and evaluates horizons h ∈ {1, 2, 4, 8}.Training uses 400 sequences and testing uses 160 sequences, each of length 80.
  • Models: MCJEPA uses one trainable 4 × 4 row-normalized transition matrix, while the baseline learns independent matrices for each horizon.Both JEPA variants use online and target encoders, with the target updated by exponential moving average.
  • Collapse diagnostics: State-usage diversity and assignment confidence measure different collapse properties and must be interpreted jointly.Occupancy regularization promotes broad global usage, whereas entropy regularization promotes confident assignments.
  • Recovery and prediction: In the separated-emission regime, all three methods recover the latent process well across state recovery, transition recovery, and true-state prediction metrics.The reported metrics include ARI, aligned transition error, and true-state prediction NLL.
  • Recovery and prediction: Under ambiguous emissions, the correctly specified HMM remains strongest, while shared-transition and horizon-specific JEPA models show similar predictive NLL.Their predictive similarity contrasts with their substantially different structural consistency.
  • Markov composition: MCJEPA has zero path disagreement by construction because every multi-step prediction is a power of the shared transition matrix.Independently trained horizon-specific matrices exhibit nonzero direct-versus-composed disagreement.

F.4 Experiment 3: predictive compression and Markovization

Experiment 3 studies predictive compression in a finite second-order process by exhaustively comparing deterministic representations and following a stochastic optimization trajectory.

  • Predictive representations: The four-state predictive representation removes redundant history while preserving all one-step predictive information.This representation is identified among the exact structural controls and forms the predictive pair used in the compression analysis.
  • Exact compression: All 4140 deterministic partitions of eight histories are evaluated exactly to construct the compression–prediction frontier.A deterministic encoder corresponds to a set partition of the eight possible histories; 36 partitions are nondominated.
  • Compression frontier: For every tested positive β through 0.014, the exact optimum is the four-state predictive pair.At β = 0.015 it switches to two states, and at β = 1 complete compression to one state is optimal.
  • Learned continuation: The learned stochastic continuation is compared with the deterministic global frontier rather than expected to lie exactly on it.It follows a warm-started gradient trajectory and uses a categorical encoder table to remove architectural capacity as a confound.
  • Markovization diagnostic: Residual predictability measures the held-out gain from adding the previous representation state, with positive values indicating omitted transition-relevant history.Values near zero indicate no further one-step predictive benefit, while tiny negative values are treated as finite-sample variation.

F.5 Experiment 4: HMM-style training of PIB-VJEPA

Experiment 4 compares JEPA-only, HMM+filter, and hybrid training under shared data, latent-state families, and evaluation protocols. The hybrid combines sequence-evidence training with genuine JEPA latent prediction, while HMM-style training improves over JEPA-only across reported horizons.

  • JEPA-only regime: The JEPA-only regime learns latent predictions without an observation model, then receives a post-hoc Gaussian emission model solely for evaluation.The fitted emission parameters do not backpropagate into the encoder or transition matrix.
  • HMM-style regimes: HMM+filter training learns generative parameters from sequence likelihood while training the context encoder to amortize detached exact Bayesian filtering.The filtering target updates the encoder but not the HMM parameters.
  • Hybrid regime: The hybrid shares its transition matrix between HMM sequence evidence and genuine JEPA latent prediction, while retaining sequence-evidence training for emissions and initial-state parameters.Its JEPA target remains the EMA local-target construction rather than the HMM filtering posterior.
  • Results: Both HMM-style and hybrid training improve over JEPA-only at every evaluated horizon, with the largest differences at shorter horizons and narrower gaps by h = 8.The paired comparison is descriptive, and the hybrid retains only a very small sequence-evidence gap relative to HMM-style training.
  • Results: Across all five paired runs, hybrid training improves over JEPA-only in representation, sequence, transition, filtering, and true-state prediction metrics at every evaluated horizon.The authors do not infer formal statistical significance from five seeds.
Loading 2608.13621v1…