PaperlayerView exact revision on arXiv

Source-linked AI summary

Soundwich: Video Generation with Layered and Controllable Audio

Zhuo Ning, AmirHossein Naghi Razlighi, Sagi Polaczek, Daniel Cohen-Or, Ali Mahdavi-Amiri

arXiv:2610.00691v1cs.CVcs.MMcs.SD

TL;DR

Joint audio-video generators typically produce one mixed soundtrack, unlike workflows that use separate editable tracks. Soundwich generates source-specific stems around a shared video, and evaluations report gains in temporal control, source assignment, and contextual naturalness.

  • Problem

    Joint audio-video models largely treat audio as a single mixed output, unlike practical workflows that represent speech, music, effects, and ambience as separate editable layers.

    Source: §1 · p. 1

  • Method

    Soundwich expands a pretrained joint audio-video generator into source-specific audio trajectories while retaining one shared video and coordinating information across stems.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    93.1% window success versus 15.0% for naive expansion; source-assignment accuracy rises from 82.2% to 90.0%, and contextual-naturalness first-place rankings rise from 17.3% to 76.7% with broadcast.

    Source: §4.1 · p. 7, §4.2 · p. 8

  • Takeaways & Limitations

    The synchronized stems can be independently retimed, muted, replaced, or remixed while remaining coupled to the shared video.

    Source: §1 · p. 2

    Complex scenes with many simultaneous or ambiguous sources may increase interference between layers.

    Source: §5 · p. 10

Abstract

from arXiv · show

Recent joint audio-video generative models can synthesize realistic videos with synchronized sound, but typically generate audio as a single mixed track. This limits source-level control and differs from practical audiovisual workflows, where speech, music, sound effects, and ambient sounds are represented as separate editable tracks. We introduce Soundwich, a training-free framework that transforms a frozen joint audio-video flow-matching model into a generator of multiple synchronized, independently editable audio stems coupled to a shared video. Soundwich generates separate audio stems with explicit control over their temporal activity. To keep separately generated sounds coherent, we introduce a shared scene representation that communicates global audiovisual context across stems while preserving their source-level separation. We further route cross-modal interactions between each audio stem and its corresponding visual source, improving audiovisual consistency. The resulting stems remain synchronized with the video and can be independently retimed, muted, replaced, or remixed. Experiments and human evaluations show improved temporal control, source separation, and naturalness, while enabling flexible source-level editing within coherent audiovisual generation. Code is available at https://github.com/CodyNing/Soundwich.

1 INTRODUCTION

Soundwich adapts a pretrained joint audio-video generator to produce synchronized, independently controllable source stems alongside one shared video. Its design coordinates shared scene context across stems while limiting interference and linking each stem to its visual source.

1 INTRODUCTION

  • Separate stems match practical audiovisual workflows, where speech, music, sound effects, and ambience are independently represented and manipulated.Source: §1 · pp. 1–2
  • Expanding one audio trajectory into source-specific trajectories creates a trade-off: stems need shared video context for coherence, but that path can also cause leakage or wrong visual associations.Source: §1 · p. 2
  • Stem Formation establishes each stem’s identity, content, and timing, while Scene Integration coordinates stems through shared context and corresponding visual sources.Source: §1 · p. 2
  • Soundwich is a training-free framework for generating multiple source-specific audio stems with a shared video and source-level semantic and temporal control.The stems can be independently retimed, muted, replaced, or remixed.Source: §1 · p. 2

2 RELATED WORK

Prior work spans joint audio-video generation, audio-only multi-source generation, timing control, and audiovisual grounding. Soundwich differs by producing editable stems within joint video generation and coordinating their context and visual-source links.

2 RELATED WORK

  • Joint audio-video systems generally generate a single mixed soundtrack, while Soundwich produces separate speech, music, and general-sound stems coupled to one shared video.Prior audio-only multi-track methods are predominantly music-specific; Step-by-Step V2A generates complementary tracks for a fixed video using a fine-tuned guidance model.Source: §2 · p. 2
  • Unlike existing temporal-control settings, Soundwich coordinates multiple isolated sources with a jointly evolving video while retaining each source as an editable output.Source: §2 · p. 3
  • Unlike video-to-audio methods that use a fixed input video, Soundwich grounds each output stem by restricting its attention to the corresponding entity in the generated video.Source: §2 · p. 3
  • Soundwich’s asymmetric gather-broadcast attention lets a persistent scene stem collect context from active stems and return shared acoustic information while decoded tracks remain separate.Source: §2 · p. 3

3 METHOD

Soundwich expands a frozen joint audio–video generator into separate source-specific audio trajectories while retaining one shared video. Stem Formation controls source identity and timing; Scene Integration shares context and routes interactions between each stem and its visual source.

3 METHOD

  • Soundwich expands one audio trajectory into N separately editable source stems while retaining a shared video trajectory.The framework also creates an internal scene stem for shared context.Source: §3 · pp. 3–4

3.1 STEM FORMATION

  • Per-stem classifier-free guidance steers each stem toward its requested source and away from competing content.The scene stem receives a joint scene condition to collect context across sources.Source: §3.1 · p. 4
  • Cached sound and quiet carriers schedule each source by adding activation features within requested intervals and blending toward quiet features outside them.A soft activity gate tapers at interval boundaries; using a cached quiet target reduces off-window activity more effectively than a zero target at the same suppression strengths.Source: §3.1 · p. 5, Figure 3 · p. 5

3.2 SCENE INTEGRATION

  • The scene stem gathers timing and shared context from active source stems, then returns that context during each source’s active interval without direct stem-to-stem attention.The scene stem is discarded after generation, leaving the editable source stems.Source: §3.2 · pp. 5–6
  • Entity-routed refinement links each visible entity to its matching audio stem, using entity masks and source intervals to restrict cross-modal information exchange.A2V retains quiet off-window audio conditioning, while V2A updates are gated to the source’s active interval.Source: §3.2 · p. 6

4 EXPERIMENTS AND APPLICATIONS

The experiments evaluate temporal control, source separation, and audiovisual naturalness, then isolate the roles of timing carriers, source-specific guidance, scene broadcast, and entity routing. Soundwich also supports edits to individual stems and image-to-video generation with editable audio.

4 EXPERIMENTS AND APPLICATIONS

  • Post-hoc separation of mixed tracks did not improve speech evaluation and could not enforce requested timelines, with interference remaining in the outputs.The study compares five baselines across 15 scenes and 40 sources, evaluating event timing and source content.Source: §4 · p. 7

4.1 GENERATION RESULTS

  • 93.1% window success versus 15.0% for naive expansion and 85.0% for Omni; off-window activity falls from 69.7% to 1.7%.Across the benchmark, Soundwich also places both Hospital Farewell speakers within their requested windows, unlike naive expansion, which delays and overlaps them.Source: §4.1 · p. 7, Table 1 · p. 8
  • 90.0% versus 82.2% source-assignment accuracy, while Soundwich ranks first for naturalness in 87.0% of judgments.Soundwich also achieves 100.0% full-clip sound success for unrestricted-timing sounds; listeners prefer its separation in 74.4% of judgments after ties are split equally.Source: §4.1 · pp. 7–8, Table 1 · p. 8, Table 2

4.2 ABLATION STUDIES

  • 93.3% versus 15.6% window success with carrier replay; off-window activity falls from 63.06% to 1.23%.Replay also raises order accuracy from 73.3% to 100.0%; quiet carrier features guide the model toward silence.Source: §4.2 · p. 8, Table 3 · p. 9
  • 95.8% versus 61.1% source-assignment accuracy with source-specific guidance on fully overlapping sources without timing constraints.Competing-source CLAP falls from 0.262 to 0.115, while target-source CLAP rises from 0.313 to 0.344.Source: §4.2 · p. 8, Table 3 · p. 9
  • 76.7% versus 17.3% first-place contextual-naturalness rankings with scene broadcast; overall preference rises from 18.7% to 76.0%.The scene stem provides shared context for the sources’ emotional delivery and acoustic character.Source: §4.2 · p. 8, Table 3 · p. 9
  • 95.3% correct-ownership responses and 96.0% target-activation responses with entity routing across ten human-study scenes.Without routing, a sound can activate the wrong entity while its intended visual source remains inactive.Source: §4.2 · p. 9

4.3 APPLICATIONS

  • Stem editing retimes or replaces selected sounds while retaining other audio; image-to-video outputs also support dialogue swaps and removal of individual stems.Examples include moving dialogue and a cat meow to different times, replacing one speaker’s reply, and reversing who speaks first.Source: §4.3 · p. 9

5 CONCLUSION, LIMITATIONS AND FUTURE WORK

Soundwich turns a pretrained joint generator’s single soundtrack into synchronized, editable source stems, using shared scene context and source-linked audiovisual routing. The conclusion identifies complex scenes as a constraint and presents layered outputs as potential control handles beyond sound.

5 CONCLUSION, LIMITATIONS AND FUTURE WORK

  • Soundwich produces synchronized, independently controllable sound layers from a monolithic audio output, with layers serving as handles for editing and recombining generative content.The authors suggest this idea may extend to other media whose components must remain coherent while retaining individual control.Source: §5 · p. 9
  • Using the batch dimension instantiates parallel source trajectories coupled through a shared generative process, without explicitly increasing output dimensionality.The authors propose this as a possible design principle for adding layered representations to pretrained generative models.Source: §5 · p. 9
  • Complex scenes with many simultaneous or ambiguous sources remain challenging because interference between layers may increase.The approach also inherits limitations and biases of the underlying pretrained model.Source: §5 · p. 10

B.1 GENERATION PROCEDURE

  • Each source receives a sound description, exclusions, and optional activity intervals; generation produces one shared video and a separately editable audio stem per source.Stem Formation generates the video and stems, while Scene Integration refines them together before decoding the final outputs.Source: §B.1 · p. 15

B.2 INPUTS, SHARED VIDEO, AND GUIDANCE

  • Audio guidance is source-specific while video guidance follows the complete scene description; four model passes per denoising step provide the two guidance updates.The pass count is independent of source count, although computation per pass increases with it.Source: §B.2 · p. 16

B.3 REFERENCE CAPTURE AND TIMELINE REPLAY

  • Sound and quiet reference features become reusable carriers that encourage activity inside requested intervals and suppress it outside them.Activity gates use 0.20-second linear boundary transitions; sources without specified intervals bypass carrier replay.Source: §B.3 · p. 17

B.4 SCENE GATHER AND BROADCAST

  • An internal scene stem gathers active-source context, which participating stems later read during their active intervals while retaining separate audio representations.Gathering sums active source states before projection; source self-attention remains independent, and the shared-attention update is restricted to active intervals.Source: §B.4 · pp. 17–18, Figure 5 · p. 18

B.5 ENTITY ROUTING AND AUDIO OUTPUT

  • Entity masks route audio–video information between each visible source and its corresponding stem, with video-to-audio updates active only during requested sound intervals.Audio-to-video links remain in place during quiet periods so quiet audio features can guide entity behavior; sources without valid masks receive no routed video-to-audio update.Source: §B.5 · p. 19

B.6 IMPLEMENTATION SETTINGS AND MODEL ASSETS

The implementation uses quantized LTX-2.5 and Gemma-4 weights, with different LTX-2.5 variants assigned to generation and refinement. The settings specify video resolution, denoising steps, and carrier and routing configurations.

B.6 IMPLEMENTATION SETTINGS AND MODEL ASSETS

  • INT8 quantized weights for the LTX-2.5 22B development and distilled models and Gemma-4 12B text encoder were used to fit generation within 32 GB of GPU memory.The specified hardware was an NVIDIA RTX 5090.Source: §B.6 · p. 19
  • The development model handles Stem Formation with source-specific guidance, while the distilled model handles Scene Integration at CFG 1.The authors report that high CFG scales degraded generation quality with the distilled model.Source: §B.6 · p. 19
  • Stem Formation generates 249 frames at 25 fps and 384 × 224 resolution, using 40 Euler steps and video and audio CFG scales of 3 and 7.Source: §B.6 · p. 20

A 2× latent spatial upscaler increases the video resolution to 768 × 448 before Scene Integration. Refinement uses eight Euler steps with the noise-level sequence

  • Scene Integration re-noises saved video, source-audio, and scene states at starting level 0.95, refines them together, and uses positive audio and video predictions without additional CFG.The benchmark includes raw and RMS-balanced source aggregation configurations identified in Appendix C.Source: A 2× latent spatial upscaler increases… · p. 20

C EXPERIMENTAL SETUP

C.1 SCENES AND COMPARISON DESIGN

The benchmark pairs 15 varied scenes with predefined source and timing specifications. Three paired seeds per scene were reused across the compared methods.

C.1 SCENES AND COMPARISON DESIGN

  • The benchmark contains 15 scenes and 40 sources: 27 speech and 13 musical or other sounds, with two to four sources per scene.The scenes cover ordered dialogue, timed events, continuous instruments, and overlapping sounds.Source: §C.1 · pp. 20–21
  • Scene specifications include visual descriptions, desired and excluded sounds, activity intervals, and segmentation prompts for visible sources.Source: §C.1 · p. 21, Table 6 · p. 21
  • Prompts, carrier assignments, source volumes, and scene-context participation were fixed before the three-seed comparison.Continuous instruments and selected timed animal sounds bypassed gathering and broadcast but remained editable output stems.Source: §C.1 · p. 21
  • Three seeds per scene were sampled once and reused across standard LTX-2.5, naive batch expansion, Ovi, MiniMax H3, and Soundwich.Table 6 combines the schedules and paired seeds.Source: §C.1 · p. 21, Table 6 · p. 21

C.2 COMPARED METHODS AND OUTPUT COLLECTION

The comparison includes six methods, with multiple outputs per scene for five methods and one per scene for Gemini Omni. Additional experiments examine carrier replay, source-specific guidance, and aggregation settings.

C.2 COMPARED METHODS AND OUTPUT COLLECTION

  • All six methods use the same 15 scenes; five provide three outputs per scene and Gemini Omni provides one, for 240 outputs total.Scores average evaluated outputs within each scene, then average scenes equally.Source: §C.2 · p. 21
  • Standard LTX-2.5 generates one mixed soundtrack, naive batch expansion uses separate source prompts with shared video, and Soundwich adds guidance, carrier replay, scene sharing, and entity routing.The LTX-2.5 methods use the same final video resolution, but their compared audio outputs differ in stage and whether source stems are mixed.Source: §C.2 · p. 21
  • The compared external generators Ovi and MiniMax H3 produce mixed-audio videos, while Gemini Omni contributes one mixed-audio video per scene without a recorded backend revision.The missing Gemini Omni backend revision limits exact reproduction of that baseline.Source: §C.2 · pp. 21–22

C.3 EXPERIMENT-SPECIFIC GENERATION CONDITIONS

  • Of 45 main-comparison Soundwich outputs, 39 use RMS-balanced aggregation and six use raw sums, with the six raw-sum outputs coming from two named scenes.Source: §C.3 · p. 22
  • Carrier replay variants use one paired seed per scene; source-specific guidance is evaluated on six selected concurrent-source scenes.Eligibility differs across carrier conditions for some measurements, as reported in the experiment protocol.Source: §C.3 · p. 22

C.4 ADDITIONAL EVALUATION SCENES

D EVALUATION PROTOCOLS AND VALIDATION

D.1 JOINT SPEECH AND SOUND COMPARISON

The appendix evaluates speech and sound content, timing, and source assignment with automatic metrics, then uses listener judgments to assess naturalness, timing, and separation. It also reports evaluation coverage and aggregates judgments by condition.

D.1 JOINT SPEECH AND SOUND COMPARISON

  • Missed detections count as window failures, while speech alignment exclusions remove some tracks from WER and timing evaluation.The evaluation also measures off-window activity and event order; 32 speech-source evaluations are excluded, while all 208 non-speech evaluations remain.Source: §D.1 · pp. 24–25

D.2 SOURCE-CONTENT METRICS

  • Source assignment measures whether each stem best matches its intended description; non-speech CLAP compares description agreement inside and outside requested windows.The evaluation also measures target and competing CLAP similarity for complete stems.Source: §D.2 · p. 25

D.3 HUMAN EVALUATION

  • The main listening study assesses naturalness, timeline adherence, and source separation; the component study assesses scene broadcast and entity routing.Source: §D.3 · p. 25

D.3.1 PRESENTATION AND RECORDED RESPONSES

  • The main study records 18 completed responses across 15 scenes, and the component study records 15 responses with ten naturalness and ten routing trials each.Trials use neutral method labels, shuffled scene order, and shuffled candidate labels.Source: §D.3.1 · pp. 25–26

D.3.2 MAIN COMPARISON: NATURALNESS, TIMING, AND SEPARATION

  • Listeners rank overall naturalness and timeline adherence, then choose the cleaner separation between the two multi-stem methods, with ties allowed.The comparison includes Standard LTX-2.5, naive batch expansion, and Soundwich.Source: §D.3.2 · p. 26

D.3.3 COMPONENT COMPARISON: SCENE BROADCAST AND ENTITY ROUTING

  • The component study asks whether intended visible sources activate at the right times and whether unrelated entities remain inactive.Naturalness trials compare Stem Formation audio paired with full-method video, Scene Integration without broadcast, and the full method.Source: §D.3.3 · p. 26

D.3.4 AGGREGATION

  • Judgments are pooled by condition and reported as mean ranks, mean sound quality, affirmative-response percentages, and response counts.Separation preferences give half-credit for ties.Source: §D.3.4 · p. 26

D.4 VALIDATION AND SENSITIVITY OF THE AUTOMATIC EVALUATION

The appendix checks how alignment errors, unavailable speech, event placement, and containment choices affect automatic evaluation. Soundwich remains highest in the reported sensitivity comparisons, though some scene-level differences are uncertain.

D.4 VALIDATION AND SENSITIVITY OF THE AUTOMATIC EVALUATION

  • Of 20 rejected transcripts in the listening audit, 13 were judged correct; rejection affected both erroneous transcripts and intelligible speech.Among 15 accepted controls, 14 were judged correct and one was uncertain.Source: §D.4 · pp. 26–27, Table 10 · p. 27
  • Activity scores can be distorted when extraction removes target sounds or residual sounds resemble target activity.The appendix interprets activity scores alongside CLAP assignment and human source-identity judgments.Source: §D.4 · p. 27
  • Soundwich’s window success is 89.4% on the 22/40-source common subset, versus 75.0% for Omni and 34.3% for MiniMax H3.Soundwich ranks highest when rejected-track events count as failures and on the common subset.Source: §D.4 · p. 27, Table 11 · p. 27
  • Soundwich has 0.35 s onset error and 0.663 IoU, versus Omni’s 0.89 s and 0.469; timed-sound success is 70.0% versus 20.0%.Both methods reach 100.0% success for full-clip sounds, so those scores hide the timed-sound difference.Source: §D.4 · pp. 27–28, Table 12 · p. 28
  • 77.8% strict window success for Soundwich versus 64.4% for Omni; with carriers, strict success is 75.0% versus 15.6% without carriers.Under 100% containment, Soundwich remains highest and carriers retain their benefit.Source: §D.4 · p. 28, Table 12 · p. 28
  • Soundwich minus Omni window-success difference is 8.1 percentage points (95% CI [−2.8, 22.8]), while onset error differs by −0.541 s (95% CI [−1.323, −0.068]).The window-success interval includes zero; the onset-error interval supports lower error for Soundwich.Source: §D.4 · p. 28

E.1 GENERATION COMPARISON

The generation comparisons assess source content, timing, and listener judgments, then test quiet carriers, activation placement, and source-specific guidance. The results favor direct stem generation and show how design choices affect silence, timing, and leakage.

E.1 GENERATION COMPARISON

  • Soundwich achieves 90.0% source-assignment accuracy versus naive expansion’s 82.2%; across 270 main-study separation choices, 159 favored Soundwich, 27 naive expansion, and 84 tied, yielding 74.4% versus 25.6% preference with half-credit per tie.With the additional scenes, assignment is 91.6% versus 82.9%, a paired gain of 8.7 points (95% CI [2.7, 14.4]). Soundwich receives 87.0% of first-place audiovisual-naturalness rankings and 95.2% of first-place timing rankings.Source: §E.1 · pp. 28–29
  • Direct stem generation achieves 0.060 requested-window WER, 93.1% window success, 1.7% off-window activity, and 90.0% source assignment, outperforming post-hoc separation on each measure.The comparisons are on 15 scenes; the single-track baselines are separated with SAM Audio.Source: §E.1 · pp. 28–29, Table 13 · p. 29

E.2 QUIET CARRIER VERSUS ZERO SUPPRESSION TARGET

  • Quiet carriers keep requested-silence intervals quiet, while zero-target outputs contain audible random noise; carriers also reduce off-window power and improve the reported content measures.The paired comparison uses 15 scenes.Source: §E.2 · p. 29, Table 14 · p. 29

Hospital farewell

  • Figure 6 compares temporal control, source content and visible-speaker activity, and naturalness and scene fit in Hospital farewell.Source colors link visible entities to audio stems; shaded intervals mark requested speaking windows.Source: §E.1 · p. 29, Figure 6 · p. 30

E.3 ACTIVATION PLACEMENT WITHIN AN ALLOWED WINDOW

  • Late emphasis delays mean first onset by 4.018 s relative to early emphasis and yields no repetitions in 9 outputs, versus 3/9 with uniform and 2/9 with early emphasis.Late emphasis can remove early repetitions but biases speech toward the window’s end; the authors retain uniform activation to avoid an early/late preference.Source: §E.3 · p. 29, Hospital farewell · p. 31, Table 15 · p. 31

E.4 SOURCE-SPECIFIC GUIDANCE FOR SOURCE SEPARATION

  • Across eight identified leakage cases, source-specific guidance lowers similarity to the intruding source; scene-level competing CLAP falls while target CLAP and assignment improve.The cases span six scenes, including orchestra and applause, saxophone and other quartet instruments, and jackhammer noise and other construction sounds.Source: §E.4 · p. 31, Table 16 · p. 31

E.5 SCENE BROADCAST RESULTS

The appendix evaluates scene broadcast as a way to improve interaction between separately generated voices, and tests direct cross-stem attention against an intermediate scene stem. Broadcast improves listener judgments, while direct sharing blurs source content.

E.5 SCENE BROADCAST RESULTS

  • Broadcast improves mean context rank from 2.04 to 1.27, overall preference, and sound quality.Both integration variants improve quality over Stem Formation audio, and broadcast adds a further gain.Source: §E.5 · p. 32, Table 17 · p. 32

E.6 DIRECT SHARING BETWEEN SOURCE STEMS

  • Direct sharing makes stems nearly identical and mixes the two requested utterances; embedding similarity rises by 0.458 on average.Without sharing, the two transcripts remain distinct; with sharing, both contain mixed words.Source: §E.6 · p. 32, Table 18 · p. 32
  • The gather/broadcast design uses shared temporal coordinates to copy source structure into a scene stem, then scene structure back, while retaining separate output-stem attention.Source: §E.6 · p. 32

E.7 ENTITY ROUTING EVALUATION

Soundwich routing improves the timing and ownership of sound-to-entity associations, and transfers to Ovi and MiniMax H3 with source-level controls and editing. Runtime and memory rise with stem count, with five stems exceeding Scene Integration memory.

E.7 ENTITY ROUTING EVALUATION

  • All 15 responses affirmed routed ownership in Studio introduction, versus 3 of 15 without routing.Across ten scenes, listeners judged target activation and correct ownership.Source: §E.7 · p. 32, Table 19 · p. 33

F.1 CROSS-BACKBONE TRANSFER TO OVI

  • Ovi produced separate speech stems and reproduced requested utterances in order across three conversation scenes, demonstrating transfer to a second joint video-audio backbone.The transfer retained source descriptions, excluded competing sounds, and activity intervals.Source: §F.1 · p. 33

F.2 CROSS-BACKBONE TRANSFER TO MINIMAX H3

  • MiniMax H3 supports source-separated generation, timeline control, entity-routed refinement, retiming, and content replacement, with shared video features conveying context across sources.Qualitative inspection found contextually appropriate delivery and broadly consistent acoustic character.Source: §F.2 · p. 33

F.3 INDEPENDENT TIMELINE AND STEM EDITING

  • Independent stems allow sound timing or content to change while retaining other generated audio; video refinement then updates video latents with edited audio held fixed.Retiming rearranges saved audio tokens, while replacement regenerates only the selected stem.Source: §F.3 · p. 34, Table 20 · p. 34

F.4 EFFICIENCY

  • Runtime rose from 106.6 to 184.7 seconds and peak allocation from 20.7 to 28.9 GiB across one to four stems; five exceeded Scene Integration memory.Measurements are medians of three runs on one RTX 5090; configurations with two or more sources include an internal scene stem.Source: §F.4 · p. 34

F.5 OBSERVED FAILURE MODES

Soundwich has limits in temporal activation, shared-context handling, spatial ownership, and visual refinement. Stronger controls can introduce artifacts, and five output stems exceed Scene Integration memory.

Timing and source activation

  • Weak or absent activation can depend on the seed and activation-carrier match; stronger activation may cause robotic speech or audible carrier artifacts.Very short windows or complex activity sequences can reduce timing reliability, causing omissions or incorrect event order.Source: §F.5 · p. 34

Shared context

  • Persistent overlapping sources such as music can dominate scene-stem context, reducing shorter events’ contribution to broadcast context.Excluding the dominant source from gathering and broadcast lets the scene stem focus on other sounds, while its independent stem remains in the final mix.Source: §F.5 · pp. 34–35

Efficiency

  • Five output stems exceed memory during Scene Integration, limiting this configuration to four.Figure 7 reports runtime and peak allocated GPU memory by output-stem count on one RTX 5090.Source: Figure 7 · p. 35

Spatial ownership and revision

  • Small or occluded entities may have incomplete masks; stronger A2V attention can cause visual discontinuities, and editing refinement may alter appearance or motion beyond the revised event.Local audio edits can therefore have broader visual consequences.Source: §F.5 · p. 35
Loading 2610.00691v1…