Source-linked AI summary
Segment Length Matters: A Study of Segment Lengths on Audio Fingerprinting Performance
Ziling Gong, Yunyan Ouyang, Iram Kamdar, Melody Ma, Hongjie Chen, Franck Dernoncourt, Ryan A. Rossi, Nesreen K. Ahmed
TL;DR
Audio fingerprinting commonly uses fixed-duration segments, but the effect of segment length has remained underexplored. This paper modifies a neural fingerprinter to test segment lengths across query durations and finds that short segments generally perform best, while GPT-5-mini most consistently recommends effective choices.
Problem
The effect of segment length on audio fingerprinting performance remains unexplored, despite segmentation being widely used in audio tasks.
Method
The paper extends NAFP with NAFP+ to evaluate multiple segment lengths and query durations on a music audio fingerprinting retrieval task.
Results
Shorter segments, especially 0.5 seconds, generally perform best, while GPT-5-mini provides the strongest segment-length suggestions among the three studied LLMs.
Takeaways & Limitations
The findings provide practical guidance for selecting segment duration in large-scale neural audio retrieval systems.
Abstract
from arXiv · showhide
Audio fingerprinting provides an identifiable representation of acoustic signals, which can be later used for identification and retrieval systems. To obtain a discriminative representation, the input audio is usually segmented into shorter time intervals, allowing local acoustic features to be extracted and analyzed. Modern neural approaches typically operate on short, fixed-duration audio segments, yet the choice of segment duration is often made heuristically and rarely examined in depth. In this paper, we study how segment length affects audio fingerprinting performance. We extend an existing neural fingerprinting architecture to adopt various segment lengths and evaluate retrieval accuracy across different segment lengths and query durations. Our results show that short segment lengths (0.5-second) generally achieve better performance. Moreover, we evaluate LLM capacity in recommending the best segment length, which shows that GPT-5-mini consistently gives the best suggestions across five considerations among three studied LLMs. Our findings provide practical guidance for selecting segment duration in large-scale neural audio retrieval systems.
I. INTRODUCTION
The paper addresses the previously unexplored effect of segment length on audio fingerprinting performance. It evaluates modified neural fingerprinting models and finds that short segments perform better, while GPT-5-mini gives the best recommendations among the studied LLMs.
- Audio fingerprinting has not previously examined how segment length affects model performance.
- The study investigates audio identification by segmenting database recordings and queries, then retrieving closest fingerprinted segments.
- The authors extend the 1-second NAFP architecture with NAFP+ to support different segment lengths.
- For queries shorter than 3 seconds, 0.5-second segments achieve significantly better performance.
- GPT-5-mini provides the best segment-length suggestions among GPT-5-mini, Gemini-2.5-flash, and Claude-Sonnet-4.5.
II. EXPERIMENTS WITH NAFP AND NAFP+
This section introduces the experimental setups using the original NAFP model and the proposed NAFP+ variant.
- The experiments use both the original NAFP model and the proposed NAFP+ variant.
A. Preliminary
The evaluation represents audio collections as reference and distractor sets, derives queries from the reference set, and segments each query using a configurable window and hop.
- The evaluation collection combines a reference set Dref with a distractor set Ddist.
- Queries derived from Dref are segmented with a moving window of size W and hop h over query length L.
- The number of resulting query segments is denoted S, producing q1, q2, …, qS.
B. NAFP+ Design
NAFP+ supports multiple segment lengths by converting each segment to a mel-spectrogram and normalizing variable frame counts before the convolutional blocks.
- B. NAFP+ Design: Training segments use W ∈ {0.5, 1, 2} seconds with a fixed hop of 0.5 seconds.
- B. NAFP+ Design: Each segment is transformed into a mel-spectrogram with 256 user-selected Mel-frequency bins.
- B. NAFP+ Design: For W values of 0.5, 1, and 2 seconds, the corresponding frame counts T are 16, 32, and 63.
- B. NAFP+ Design: NAFP+ adds a fully connected ELU layer before each convolutional block to normalize T to a fixed T0 = 32.
C. Evaluation
The evaluation identifies original audio from reference and distractor sets by segmenting queries into overlapping windows and aggregating nearest-segment retrievals. It reports Hit Rate across segment and query lengths on a derived music dataset.
- Evaluation setup: The task retrieves the correct original audio from a reference set within a larger dataset containing unrelated distractors.Queries are derived from reference audio, while distractors prevent retrieval from becoming trivial.
- Query processing: Queries select an L-second subaudio, then apply W-second windows with hop h to produce S query segments.Each resulting segment is matched independently before query-level aggregation.
- Retrieval: Each query segment retrieves closest database segments, and weighted segment majority voting selects the corresponding audio.
- Dataset: The dataset contains 10,000 training clips, 500 reference clips, and 9,978 distractor clips, with 30-second musical pieces.Queries are synthesized from reference clips using held-out background noises, time offsets, and room impulse responses.
- Metrics: The evaluation reports Top-K Hit Rate across query segments, combining Top-K Exact Hit for K ∈ {1, 3, 10} and Top1-Near Hit.Near Hit permits an extra ±1 frame misalignment, and higher Hit Rate is more favorable.
III. RESULTS
Across query lengths, 0.5-second segments generally perform best, especially for short queries, while longer queries reduce the gap between 0.5- and 1-second segments. Performance rises quickly with additional segments and largely saturates after 4 seconds.
- Segment-length results: 0.5-second segments achieve the best performance on most query lengths, exceeding an 8 / 10 win rate in every metric group.Two-second segments perform worst across query lengths.
- Segment-length results: For query lengths L ≤ 3, shorter segments consistently achieve significantly higher Hit Rates across metrics.
- Segment-length results: Beyond 3-second queries, the performance gap between 0.5- and 1-second segments narrows substantially.The passage attributes this pattern to additional temporal context compensating for coarser segmentation.
- Query-length results: Performance improves rapidly with additional segments, begins to saturate after 4 seconds, and shows only slight improvement afterward.
A. Segment Length Selection with LLMs
The study evaluates whether LLMs can recommend segment lengths using five prompts that vary the decision considerations. It compares recommendations from three models against the empirical segment-length findings.
- LLM evaluation: The evaluation compares GPT-5-mini, Gemini-2.5-flash, and Claude-Sonnet-4.5 across five prompts for selecting the segment length with the highest Hit Rates.
- Prompt considerations: The five prompts consider temporal resolution and discriminative power, robustness, accuracy-speed-storage trade-offs, and variable-duration query generalization.
- Prompt design: The prompts combine a fixed context describing the task and dataset with one question prompt for each consideration.
X Prompt Template
The prompt template combines fixed task and dataset context with one of five question prompts. These prompts ask LLMs to reason about segment duration in a neural fingerprinting system that produces one embedding per segment.
- Prompt context: Segment duration is the fixed-length audio-segment duration, measured in seconds, used by the contrastive neural fingerprinting model.
- Prompt context: The model generates one embedding per segment for similarity-based matching in a large-scale music database.
- Prompt template: Figure 3 combines a fixed context prompt with one of five question prompts.
- Prompt template: The questions address temporal resolution, robustness, retrieval accuracy-speed-storage trade-offs, and generalization to variable-duration queries.
- LLM comparison: GPT-5-mini consistently recommends approximately 1-second segments, aligning more closely with the empirical optimum than the other models' longer suggestions.
IV. CONCLUSION
The study finds that segment length affects audio fingerprinting performance, while performance gains saturate after 4-second queries. GPT-5-mini consistently recommends a 1-second segment length, and broader generalization remains a future research direction.
- Performance gains saturate after 4-second queries regardless of the selected segment lengths.
- GPT-5-mini consistently recommends a 1-second segment length across different prompts, aligning closely with empirical results.
- Gemini-2.5-flash and Claude-Sonnet-4.5 show higher variability and tend to suggest longer segment durations.
- Future work should examine whether these findings generalize beyond music to speech and environmental sounds.