An Exploratory Study of Frequency-Aware Task Weighting for YOLOv8-Based Unified Driving Perception

• Separate networks for detection, drivable-area segmentation, and lane segmentation can increase redundant computation, memory consumption, and deployment complexity.Source: §1 · p. 1
• Task losses can differ in scale, convergence, and optimization behavior, so a weighted sum may let one task dominate and degrade others.Source: §1 · p. 2
• Existing approaches draw weighting signals from loss magnitudes, learned uncertainty, short-term loss changes, or gradient information; this work explores longer loss-history frequency structure as a complementary signal.Source: §1 · p. 2
• FTW weights tasks according to the low-frequency power ratio in recent loss trajectories, using a short loss history without extra weighting parameters or task-specific gradient computations.Source: §1 · p. 2
• The study compares fixed, uncertainty-based, and FTW weighting with static or progressive scripts in a unified YOLOv8 framework on Mapillary Vistas.Source: §1 · p. 2

2.1 Unified Driving Perception

• Shared-feature systems such as MultiNet, YOLOP [4], YOLOPv2 [5], and HybridNets [6] produce multiple driving-perception outputs while reducing redundant computation.Source: §2.1 · p. 2

2.2 Multi-task Optimization

• A weighted multi-task objective can let one task dominate shared representations or cause negative transfer when task improvements conflict.Source: §2.2 · p. 3

2.3 Adaptive Task Weighting

• Fixed and uncertainty-based weighting are the implemented baselines; FTW instead summarizes recent loss-window frequency structure, without claiming to be the first spectral or temporal approach.Source: §2.3 · p. 3

2.4 Stability-aware Optimization

• Loss trajectories may fluctuate during optimization, motivating a stability-aware weighting investigation, but these experiments do not validate the chosen spectral statistic as a stability measure.Source: §2.4 · p. 3

• For an input image, the network predicts object detections, a drivable-area mask, and a lane mask.Source: §3.1 · p. 3
• The overall objective is a weighted sum of task-specific losses, with N = 3 for detection, drivable-area segmentation, and lane segmentation.Source: §3.1 · p. 4
• The objective is to examine an adaptive rule that sets each task weight according to its recent optimization stability.Source: §3.1 · p. 4

• A shared YOLOv8 backbone and neck provide multi-scale features to detection, drivable-area segmentation, and lane-segmentation heads.Source: §3.2 · p. 4
• The detection branch uses the YOLOv8 head, while two lightweight decoder heads handle segmentation; lane loss combines binary cross-entropy and Dice loss.Source: §3.2 · p. 4
• Drivable-area segmentation uses pixel-wise cross-entropy with ignored pixels excluded, and the lane-loss coefficients are each 0.5.Source: §3.2 · p. 4
• All three tasks update the shared backbone and neck, where imbalanced or conflicting objectives may negatively affect the shared representation.Source: §3.2 · p. 4

3.3 Progressive Training Schedule

• Static training keeps all network parameters trainable throughout optimization.Source: §3.3 · p. 4
• Progressive training first trains all parameters, then freezes early backbone layers, and finally freezes the backbone and neck while updating only task-specific heads.In Stage II, the remaining backbone layers and neck stay trainable.Source: §3.3 · p. 4
• Selected progressive-training checkpoints precede every freezing transition, so the available runs cannot test the effects of progressive freezing.Progressive training is configured independently of the task-weighting rule.Source: §3.3 · p. 4

3.4.1 Motivation

• Prior weighting methods use uncertainty, short-term loss changes, or gradient information; FTW investigates loss-trajectory frequency as a complementary stability signal.Non-monotonic loss dynamics, including oscillations and spikes, motivate examining temporal behavior.Source: §3.4.1 · pp. 4–5

3.4.2 Loss-history Construction

• For each task, FTW retains at most 100 recent losses and computes from the history accumulated so far, without waiting for a full window.The history is maintained as a sliding deque.Source: §3.4.2 · p. 5

3.4.3 Frequency-based Stability Estimation

• FTW mean-centers each loss history, applies a real FFT, and uses the low-frequency share of non-DC power as an operational stability proxy.The ratio is not a direct measure of gradient stability or inter-task gradient conflict.Source: §3.4.3 · p. 5

3.4.4 Dynamic Task-weight Computation

• The implementation uses an update interval of 50 loss calls and gives tasks with larger low-frequency ratios larger weights.All three tasks begin with weight 1/3; spectral ratios and weights are recomputed at each update.Source: §3.4.4 · pp. 5–6

3.4 Frequency-aware Task Weighting

• The overview traces recent task-loss windows through frequency analysis to stability proxies and then dynamic weights in the multi-task objective.Source: Figure 1 · p. 6
• FTW derives task weights directly from recent loss trajectories, without additional optimized weighting parameters or task-specific gradient statistics.Its effective implementation uses loss-history buffers and frequency-domain computations.Source: §3.4.4 · p. 6

4.1 Dataset and Preprocessing

• Mapillary Vistas data include 18,000 training images and a validation split divided into separate 1,000-image checkpoint-selection and final-holdout subsets.The dataset contains 25,000 high-resolution street-scene images; the shared preprocessed labels cannot be independently reconstructed because conversion scripts and exact category mappings are absent.Source: §4.1 · p. 7

4.2 Implementation Details

• The shared YOLOv8-based model is trained for 80 epochs with a global batch size of 32, and the six configurations are single logged runs.Current scripts specify different training seeds for static uncertainty weighting than for the fixed, FTW, and progressive-uncertainty runs, but historical logs do not verify the seeds actually used.Source: §4.2 · p. 7
• The progressive schedule trains all parameters through epoch 47, partially freezes the backbone for epochs 48–71, then freezes the backbone and neck for epochs 72–80.Source: §4.2 · p. 7
• Table 1 compares final-holdout metrics for three weighting strategies under static and progressive schedules, using each run’s minimum-validation-loss checkpoint.The table caption identifies the comparison but does not provide its cell values in the supplied passage.Source: §4.2 · p. 7, Table 1 · p. 8

4.3 Compared Weighting Strategies

• The comparison uses equal fixed weights, learnable uncertainty-based weighting, and FTW’s dynamic low-frequency-ratio weighting.Uncertainty weights are unnormalized and updated separately on each GPU process; the logged FTW implementation likewise maintains separate histories and weights per process.Source: §4.2 · p. 7, §4.3 · p. 8

4.4 Evaluation Metrics

• Detection is evaluated with mAP50 and mAP50:95, segmentation with drivable-area and lane mIoU, and the reported overall score weights mAP50:95, drivable mIoU, and lane mIoU.The overall score is calculated from final-holdout metrics and is not used for checkpoint selection; differing validation objectives mean selected minima are not a common numerically comparable criterion across weighting strategies.Source: §4.4 · p. 8

4.5 Quantitative Results

• Table 1 reports final-holdout metrics for each run’s minimum-validation-loss checkpoint, and Figure 2 visualizes task-specific performance.Source: §4.5 · p. 8, Figure 2 · p. 9

4.5.1 Comparison of Weighting Strategies

• Static FTW led the derived overall score (0.4135) and lane mIoU (0.6403), progressive FTW led detection mAP50:95 (0.0593), and static uncertainty led drivable-area mIoU (0.8688).These are rankings among the six logged runs.Source: §4.5.1 · pp. 8–9, Figure 2 · p. 9
• Fixed-weight runs had similar detection and drivable-area scores, while progressive-labeled fixed weighting raised lane mIoU from 0.4589 to 0.5276 and overall score from 0.3763 to 0.3891.Across all metric columns, the largest observed values came from different configurations, and one run per configuration does not establish method-level superiority.Source: §4.5.1 · p. 9

4.6 Comparison of Static and Progressive Configurations

• All six selected checkpoints came from epochs 6–11, before progressive freezing began at epoch 48, so static–progressive differences do not test the freezing operations.They compare logged trajectories before the schedules diverged.Source: §4.6 · p. 9

4.7 Training Dynamics Analysis

• Figure 3 shows training and validation loss curves, but those traces describe convergence and fluctuations without identifying their causes.Source: §4.7 · p. 9, Figure 3 · p. 10
• Figure 4’s normalized task-loss curves use every-tenth-batch rank-0 records, not full-batch epoch averages, and cannot reconstruct the process-local histories that generated FTW’s weights.The available traces therefore do not test FTW’s stability mechanism.Source: §4.7 · pp. 9–10, Figure 4 · p. 10

5.1 Optimization Stability as a Task-balancing Signal

• FTW treats recent task-loss trajectory frequency as an additional stability signal, complementing approaches based on uncertainty, loss changes, or gradient statistics.The study leaves open whether this signal improves performance or optimization stability.Source: §5.1 · pp. 10–11
• The completed runs show FTW can be implemented in the unified pipeline, but their metric leaders differ by task and represent one run per configuration, not expected performance.Static FTW led overall score and lane mIoU, progressive FTW led detection mAP, and static uncertainty weighting led drivable-area mIoU.Source: §5.1 · p. 11

5.2 Relationship to Existing Weighting Methods

• FTW’s low-frequency ratio differs from the signals used by the implemented baselines, but no combined-method experiment tests compatibility or benefit.The paper frames combining it with uncertainty- or gradient-based approaches as a future hypothesis.Source: §5.2 · p. 11

5.3 Practical Implications

• The logged FTW configuration adds a loss-history buffer and frequency analysis without extra weighting parameters or task-specific gradient manipulation, but computational overhead was not benchmarked.Process-local state and validation-history contamination make the current implementation a research prototype rather than a production-ready distributed component.Source: §5.3 · p. 11, §5.4 · p. 12

5.4 Limitations and Future Work

• Evidence is limited to one network family, one dataset, and one run per configuration, without repeated seeds, uncertainty estimates, or single-task baselines; generalization beyond this driving pipeline is unevaluated.The static-uncertainty seed differs in the current scripts, and historical run seeds cannot be verified.Source: §5.4 · p. 11, §5.3 · p. 11
• The experiments do not directly report FTW’s task weights, low-frequency ratios, or complete loss histories, and no ablations test its window, update interval, or frequency boundary.Loss oscillations also do not directly measure gradient conflict, gradient noise, or negative transfer.Source: §5.4 · pp. 11–12
• The archived materials lack upstream annotation-conversion scripts and exact category mappings, and the selected checkpoints precede progressive freezing.The reported results therefore cannot evaluate the interaction between freezing and task weighting.Source: §5.4 · p. 12

6 Conclusion

• FTW estimates task-specific stability from low-frequency energy ratios of recent, mean-centered loss trajectories and uses them to construct a dynamic multi-task objective.Source: §6 · p. 12
• Static FTW led derived overall holdout score and lane mIoU, progressive FTW led detection mAP, and static uncertainty weighting led drivable-area mIoU among six minimum-validation-loss checkpoints.Source: §6 · p. 12
• With one logged run per configuration and other implementation and reproducibility limits, the rankings do not establish an FTW advantage; the supported conclusion is feasibility in this pipeline, not proven stability improvement.Source: §6 · p. 12