PhysicsSliders: Learning Physical Controls for Audio Foley Generation

1. INTRODUCTION

• Material, geometry, and excitation are natural controls because they shape attack, resonances, and decay, and could adapt Foley to changing scenes.The passage names games, virtual environments, animation, editing alternatives, and room acoustics as potential uses.Source: §1 · p. 1
• Prior controls target events and scenes [1], timing, acoustic trajectories, latent attributes, and semantic edits [6], while physics-aware work conditions object and interaction properties [9].The introduction distinguishes those control approaches from meeting specified acoustic targets.Source: §1 · p. 1
• Target-driven Foley generation must meet specified acoustic values, including two at once, and evaluate absolute accuracy alongside response direction.The introduction identifies object properties needed for joint control as an open question.Source: §1 · p. 1
• PXS introduces direct-parameter and numerical-caption interfaces, evaluates simultaneous damping and frequency targets, and maps real-recording damping targets to generation inputs without real-audio fine-tuning.It also tests whether thickness information improves joint control and studies control transfer to held-out materials, room descriptions, and spectrally distinct outputs at fixed targets.Source: §1 · p. 1

2. METHOD

• Separate PXS variants share a flow-matching architecture and frozen decoder: PXS-Physical accepts parameters with or without text, while PXS-Text accepts text alone.The model uses an eight-layer, 54.8-million-parameter transformer with a frozen pretrained Oobleck VAE; physics and text embeddings condition generation.Source: §2 · pp. 1–2
• Online modal simulation generates plate and bar sounds from 15 physical parameters, Hertzian impacts, and randomized surface excitation, rooms, microphone response, noise, and gain.The parameters cover material and damping, geometry, excitation, and motion; room decay time spans 0.08–0.8 s.Source: §2 · p. 1
• Training varies size, thickness, hardness, loss, speed, and damping across specified ranges, using six training materials and holding out aluminium, brass, and concrete.Aluminium overlaps training support, while brass and concrete lie between training bands.Source: §2 · p. 2
• The model adds shape and contact-type tokens to parameter tokens, pooling all 17 tokens for conditioning.PXS-Physical drops text, physics, and individual coordinates during training; PXS-Text always masks physics.Source: §2 · p. 2
• Effective damping q is the median of valid decay readings, but excitation, rooms, modal frequencies, and κ mean q need not equal source loss η.Readings are derived from fitted decay rates and frequencies; invalid readings are omitted, and q is missing if no peak is valid.Source: §2 · p. 2
• Simulator inversion maps template losses to effective-damping targets using 24 log-spaced loss values and three excitation/recording realizations per loss.Template loss specifies the generation input for each target; templates use 0.30-m plates and class-assigned materials.Source: §2 · p. 2

(a) Physical and numeric-text control

• PXS-Physical lists .976, .77, .094, .829, .19, and .293; PXS-Text lists .954, .78, .058, .844, .27, and .360.The supplied row passage does not identify the metric associated with each value.Source: (a) Physical and numeric-text control · p. 3
• The encoding ablation reports model/simulator slope ratios across smooth, high-frequency, and direct encodings, with 1 as the ideal ratio.The table caption identifies two runs and marks PXS-Physical in bold.Source: Table 2 · p. 3
• The two-second zero-shot grouped-rank evaluation scores incomplete class responses as zero.The caption specifies one-sided class-paired Wilcoxon signed-rank tests, Holm-corrected across eight comparisons.Source: Table 3 · p. 3

3. EXPERIMENTS

• Evaluation covers 18 impulsive and 12 sustained classes, with 240 requests including targets outside simulator support; class-specific calibrated thresholds set ranking resolution rather than target tolerance.The study evaluates classes from ESC-50, FSD50K, AudioSet, NSynth, and RealImpact.Source: §3 · p. 3

3.1. Target delivery and physical responses

• 0.77 (95 % class-bootstrap confidence interval [0.65, 0.86]): PXS-Physical gain for real-impact damping targets; both learned interfaces track them, while sustained responses cover less of the target range.For measurement validation, hc yields 0.941 ungrouped correlation and 78.9 % 1.5× request-median agreement, versus 0.986 and 40.7 % for the alternative readout; ordering is stable, but absolute agreement depends on the decay definition.Source: §3.1 · p. 3, §3 · p. 3
• 323 of 432 request–run evaluations yield 1.5×-accurate pairs across PXS-Physical and both simulator arms; class-median distances are 10.27 dB, 7.15 dB, and 15.03 dB, respectively.The three distances correspond to PXS-Physical, fixed-aspect simulation, and varied-aspect simulation.Source: §3.1 · p. 3
• PXS-Physical’s speed and κ response ratios span 0.72–1.41 and 0.89–1.04, respectively; smooth encoding or the direct path preserves hardness and damping responses, while removing both weakens them.Five of six material–contact combinations qualify for speed and four of six for κ.Source: §3.1 · p. 4

3.2. Conditioning interfaces

• SAO’s impact-class correlations are -0.085 for q and -0.158 for η∗; TangoFlux completes 14/18 numeric and 10/18 ordinal classes, with complete-class mean correlations -0.534 and 0.535.These results use predefined 2-second energy windows from 10-second outputs.Source: §3.2 · p. 4
• 0.171 versus 0.440 dex damping error favors PXS-Physical over SonicGauss on four ObjectFolder-Real objects, while frequency error favors SonicGauss, 0.366 versus 0.520 dex.PXS-Physical uses reference damping and proxy plates; SonicGauss receives geometry and contact coordinates.Source: §3.2 · p. 4

3.3. Joint control

• 68.0 % versus 15.6 % joint success favors PXS-Physical over PXS-Text across 225 setting–run evaluations; masking thickness lowers success to 27.6 %, while supplying it raises success at tested h/L settings.With 32-output simulator medians, supplying thickness raises 1.5× joint success from 51.1 % to 80.0 % and from 57.8 % to 95.6 % at h/L = 0.016 and 0.032.Source: §3.3 · p. 4
• Reverberant versus dry room text raises tail-to-attack energy in all 30 comparisons, with context medians of 5.75–10.10 dB, while all 18 size slopes remain negative.Frequency-scale shifts across the size slopes are +0.097–0.240 dex.Source: §3.3 · p. 4

• PXS delivers effective-damping targets from real impacts without real-audio fine-tuning through direct-parameter and numerical-caption interfaces.Source: §4 · p. 4
• Specifying thickness improves frequency accuracy and joint delivery when controlling damping and frequency targets together.Source: §4 · p. 4
• Parameter inputs can be combined with room descriptions, and repeated sampling produces spectrally distinct sounds at fixed targets.Source: §4 · p. 4
• Future work targets sustained range compression, cross-talk, and perceptual evaluation.Source: §4 · p. 4