PaperlayerView exact revision on arXiv

Source-linked AI summary

Converting Sequenced Fuzzy Cognitive Maps to Causal Virtual Worlds with Large Video Generators

Akash Kumar Panda, Olaoluwa Adigun, Bart Kosko

arXiv:2609.14985v1cs.AIcs.CLcs.IR

TL;DR

The paper addresses how to turn sequences of FCM states into causal video scenes. It sequences FCM stimuli and equilibria, connects them to scripted scene descriptions, and reports an eight-scene dolphin-and-shark story in which the dolphins face a persistent threat.

  • Problem

    The paper addresses how to represent a sequence of FCM stimuli and equilibria as a video storyboard.

    Source: F. FCM Sequencing · p. 5

  • Method

    FCM stimulus patterns drive transitions between equilibria, which an LLM agent annotates as a script for corresponding video scenes.

    Source: F. FCM Sequencing · p. 5, A. Semi-autonomous agentic control · pp. 7–8

  • Results

    Eight scenes combine limit-cycle and fixed-point equilibria into a story of dolphins chased by a persistent shark and pushing through fatigue to escape.

    Source: F. FCM Sequencing · p. 5

  • Takeaways & Limitations

    Sequenced FCMs can combine with agentic control of large video generators to produce causal virtual worlds, with larger versions offering a way to design immersive worlds for multiple users.

    Source: §IV · pp. 8–9

Abstract

from arXiv · show

We show how users can create and manipulate causal virtual worlds with large-language-model (LLM) and large-video-model agents. The approach uses feedback fuzzy cognitive maps (FCMs) both to model the granular causal structure of the virtual world and to guide its causal evolution. The local causal rules are partial or fuzzy while the FCM's feedback structure produces global equilibria that define causal scenarios. A sequence of \emph{dynamical} meta-rules of the form ``If $\mathcal{A}$ then $\mathcal{B}$" define the causal scenes of the virtual-world video. The if-part causal pattern $\mathcal{A}$ perturbs the FCM's virtual world at the user's or agent's discretion. The FCM's transient feedback dynamics define the meta-rule's causal arrow of implication. The then-part $\mathcal{B}$ is the resulting equilibrium attractor such as a FCM limit cycle or fixed point. Our algorithm extracts these meta-rules from the FCM and guides the LLM agent to write a script based on the FCM meta-rule sequence. The large video generator converts the meta-rule into a video scene in accord with the flow of the dynamics. We applied the agent-based technique to a simple FCM that describes an undersea world of dolphins and sharks. Google's Gemini 3.1 generated the script and Google's Veo 3.1 generated the dolphin-shark video. The approach is general and can scale by mixing larger FCMs and AI agents to produce more immersive virtual worlds.

A. Fuzzy Cognitive Maps (FCM)

Fuzzy Cognitive Maps (FCMs) represent causal dynamical systems as directed weighted graphs, using feedback to model how inputs lead to equilibria.

A. Fuzzy Cognitive Maps (FCM)

B. Causal Edge Matrix

An FCM’s causal edge matrix encodes the direction and strength of relationships between concept nodes; each matrix entry corresponds to a directed edge.

B. Causal Edge Matrix

  • An edge eij from concept Ci to Cj means that Ci causes Cj.Source: B. Causal Edge Matrix · p. 2
  • The weight eij ranges from -1 to 1, with its sign indicating whether Cj increases or decreases as Ci increases.A zero weight means no edge; a larger magnitude indicates a stronger causal relationship.Source: B. Causal Edge Matrix · p. 2
  • An n×n matrix E stores the edge weights, with eij in row i and column j and zero entries for absent edges.In the dolphin FCM, the Survival Threat-to-Run Away entry e45 is 1, representing a positive edge.Source: B. Causal Edge Matrix · p. 2

C. FCM Evolution

FCM evolution updates node states over discrete time steps, producing trajectories that converge to equilibria; the paper illustrates this with a five-node dolphin model.

C. FCM Evolution

  • A state vector records each node’s activity, with values near one active, near zero inactive, and intermediate values partially active.Source: C. FCM Evolution · p. 2
  • Figure 1 depicts an eight-scene video, with each scene’s input stimulus above its frame and the FCM states at the beginning and end.The figure highlights the clamped Herd Clustering node in green.Source: Figure 1 · p. 3
  • The next node state is computed from the current state and causal edge matrix, then compressed by a bounded nonlinear function.For the illustrated video, the function is a threshold nonlinearity.Source: C. FCM Evolution · pp. 2–3
  • Starting with Survival Threat active and other nodes off, the dolphin FCM activates Herd Clustering and Run Away in the next state.This illustrates dolphins grouping and fleeing when a shark-like threat is present.Source: C. FCM Evolution · pp. 2–3

D. FCM Equilibria

FCM equilibria include fixed points and repeating limit cycles; their basins of attraction map initial conditions to the resulting attractors.

D. FCM Equilibria

  • A fixed point is a constant state, while a K-step limit cycle repeats its states in the same order.FCMs may also converge to a chaotic attractor without a repeating state pattern.Source: D. FCM Equilibria · p. 3
  • The dolphin FCM’s four-state cycle repeats threat, clustering and escape, fatigue, and rest.In this example, the shark catches up while the dolphins rest, restarting the cycle.Source: D. FCM Equilibria · p. 3
  • A basin of attraction is the set of initial states that lead to a given equilibrium; the basins partition the input space.The FCM approximates the map from those basins to their equilibrium attractors.Source: D. FCM Equilibria · p. 3

E. Causal State Activation: Clamping vs. Pulsing

Clamping and pulsing provide different ways to perturb the dolphin FCM, producing different attractor behavior. Clamping sustains a node value, while pulsing applies it for one time step before unperturbed dynamics resume.

Causal State Activation: Clamping vs. Pulsing

F. FCM Sequencing

FCM sequencing turns ordered clamp and pulse patterns into corresponding equilibria, with each stimulus moving the system from its prior equilibrium. The resulting sequence of transitions defines the virtual-world scenes.

F. FCM Sequencing

  • An input pattern Ak perturbs the FCM, whose transient dynamics connect it to an equilibrium attractor Bk.The resulting attractor may be a limit cycle or fixed point.Source: F. FCM Sequencing · p. 5
  • The eight input-output pairs map stimuli A1-A8 to equilibria B1-B8, and each transition defines a scene.The stimulus breaks the prior equilibrium before the FCM settles into the new one.Source: F. FCM Sequencing · p. 5, Figure 4 · p. 6
  • The eight scenes tell a story of dolphins pursued by a persistent shark and pushing through fatigue to escape.The equilibria include limit cycles B1-B4 and B8, plus fixed points B5-B7; B8 repeats the B2 limit cycle.Source: F. FCM Sequencing · p. 5
  • The first scene clamps on Herd Clustering and converges to a four-stage limit cycle, with a frame and active nodes shown for each stage.The following scenes use pulsed or clamped Survival Threat conditions, with Scene 4 also clamping off Herd Clustering.Source: Figure 5 · p. 6, Figure 6 · p. 7, Figure 7 · p. 7

A. Semi-autonomous agentic control

An LLM annotates sequenced FCM equilibria as a script whose scenes express state transitions, and a video generator turns the script into a sequence of clips. The demonstrated process yields a 57-second video covering all eight scenes.

A. Semi-autonomous agentic control

  • The eight equilibria and their transitions form the storyboard, with each scene corresponding to an equilibrium state.The LLM can use the state vector sequence directly or an intermediate visualization.Source: A. Semi-autonomous agentic control · p. 7
  • The LLM script divides the movie into a task description, summary, and scene descriptions that translate state variables into visual elements, dialogue, and interactions.Scenes align with transitions within the equilibrium sequence.Source: A. Semi-autonomous agentic control · p. 7
  • Sequentially extending the generated clip through all eight scene descriptions yielded a 57-second video.The Veo 3.1 generator allows up to eight seconds per generated video, motivating the multiple-shot approach.Source: A. Semi-autonomous agentic control · p. 8
  • Scene 1 depicts dolphins playing and resting with Herd Clustering clamped on and Survival Threat inactive.The corresponding video frames represent the causal virtual world for Scenes 2-8 as well.Source: A. Semi-autonomous agentic control · p. 8

IV. Conclusions

Sequenced FCMs can guide agent-controlled video generators, while users can alter causal patterns or local FCM structure midstream. The authors propose that large-scale versions could support realistic, immersive virtual worlds with multiple interacting users.

IV. Conclusions

  • Sequenced FCMs combine with agentic control of video generators, and users can modify causal patterns or local FCM structure midstream.The authors state that propagation uses simple vector–matrix multiplication, often with sparse causal edge matrices, and that mixing FCMs always produces a new FCM.Source: §IV · p. 8
  • Large-scale versions offer, in principle, a way to design realistic, immersive virtual worlds that allow multiple users to interact.Source: §IV · p. 9
Loading 2609.14985v1…