Source-linked AI summary
BrachistoneLR: A Brachistochrone-Inspired Learning-Rate Schedule and a Controlled Benchmark of Scheduling Policies
Md. Sadekur Rahman Roni, Md. Jalal uddin Chowdhury, Moutusi Dash Nimi
TL;DR
Published schedule comparisons are difficult to interpret when architecture, dataset, and budget vary alongside the policy. The paper characterizes BrachistoneLR as a cosine-period choice and benchmarks six schedules under a fixed protocol. Smooth peak-to-floor schedules outperform constant and calendar-based decay, while the three leading smooth schedules are close in aggregate.
Problem
Published comparisons vary architecture, dataset, and budget alongside learning-rate policy, making it difficult to attribute reported differences to the schedule.
Source: §1 · p. 2Method
The paper characterizes BrachistoneLR as a brachistochrone-derived schedule equivalent to cosine annealing with half-period E −1, then benchmarks six schedules under a fixed protocol.
Source: §1 · p. 2, §3.2 · p. 4Results
Smooth peak-to-floor schedules take the top three mean ranks and finish within 0.06 points of one another, while constant and calendar-based rules trail by half a point to a point overall.
Source: §5 · p. 9Takeaways & Limitations
The E −1 terminal-rate distinction from conventional cosine annealing shrinks as E−2, making it a short-horizon effect.
Source: §6 · p. 11
Abstract
from arXiv · showhide
The learning-rate schedule is a consequential choice in training deep networks, yet the policies in common use are heuristic, and published comparisons are hard to read, because architecture, dataset, and budget tend to vary alongside the schedule. We study BrachistoneLR, a schedule built by mapping the vertical coordinate of the brachistochrone, the curve of fastest descent under gravity, onto the range between a peak and a floor rate. Expanding the definition shows it to be cosine annealing with the half-period set to E - 1 instead of E, the configuration a standard implementation gives when its period argument is one less than the number of epochs. The rate therefore reaches its floor at the last epoch trained rather than one epoch later, and we show this difference decays as E^-2, making it a short-horizon effect. We then benchmark six schedules over 72 runs on three image classification datasets (MNIST, Fashion-MNIST, CIFAR-10) and four architecture families (fully connected, convolutional, recurrent, residual), fixing the optimizer, data pipeline, and evaluation protocol so that only the schedule varies. Schedules that fall smoothly from peak to floor beat the constant rate and calendar-based decay by margins that grow with task difficulty, reaching 2.5 points of dataset mean on CIFAR-10. Within that leading group, BrachistoneLR, cosine annealing, and warmup-cosine lie within 0.06 accuracy points and 0.17 of a mean rank, which one seed per configuration cannot separate. BrachistoneLR is best on both residual networks and has the highest CIFAR-10 mean, and it sets no milestones, decay factor, warmup length, or restart period. We conclude that the shape of a schedule matters more than its parameterization, that the choice of whether to use a smooth schedule matters more than the choice among them, and that the terminal-rate distinction is worth attention only over short horizons.
1 Introduction
Learning-rate policies differ in how they balance large early updates with stable late refinement, but published comparisons vary architecture, dataset, and budget alongside the schedule. This paper characterizes BrachistoneLR’s relation to cosine annealing and benchmarks six schedules under a fixed protocol.
1 Introduction
- Learning-rate schedules must balance large early updates that help escape poor initialization with small late updates that stabilize approach to a minimum.The paper notes that both requirements matter for a useful schedule.Source: §1 · p. 2
- Published comparisons make schedule effects difficult to isolate because policies differ in shape and exposed parameters while architecture, dataset, and budget also vary.A large rate helps early optimization, while a small late rate supports stable refinement.Source: §1 · p. 2
- BrachistoneLR is a period choice within cosine annealing, obtainable in a standard library by setting the cosine period to E −1.The paper characterizes this choice and benchmarks six schedules across four architectures and three datasets under one fixed protocol.Source: §1 · p. 2
2 Related Work
Prior work developed cosine, warmup, cyclical, and other learning-rate policies, while more recent work addresses cosine annealing’s dependence on a predetermined training horizon. This paper places its contribution in controlled empirical study and an exact characterization of BrachistoneLR’s period choice.
2 Related Work
- Cosine annealing became a strong default, while linear warmup, cyclical policies, and one-cycle training developed as related approaches (Loshchilov and Hutter, 2017; Goyal et al., 2017; Smith and Topin, 2019).The cited literature also includes theoretical arguments for linear decay and iterate averaging (Defazio et al., 2023; Defazio et al., 2024).Source: §2 · p. 2
- Recent work addresses cosine annealing’s need to commit to total training length, including warmup-stable-decay (Hu et al., 2024) and a loss-landscape account of late decay (Wen et al., 2025).The paper contrasts this horizon-independence aim with BrachistoneLR’s tighter binding to E, whose deviation from conventional cosine annealing vanishes as E−2.Source: §2 · pp. 2–3
- The paper follows controlled empirical work, running every schedule with every architecture on every dataset under one protocol to separate schedule effects from setting effects.It claims no new optimization mechanism; its contribution is an exact account of the period deviation and a controlled measurement of schedule choice.Source: §2 · p. 3
3 Methodology
The methodology isolates schedule shape by holding training and evaluation fixed across a fully crossed benchmark. It defines BrachistoneLR and specifies the baseline policies, architectures, datasets, and training protocol.
3.1 Setup and notation
- Only the schedule varies across the fully crossed comparison; all runs use Adam, a 10^-3 peak rate, and a 10^-5 floor where the schedule defines one.The study covers MNIST, Fashion-MNIST, and CIFAR-10 for ten epochs, with 72 configurations in total.Source: §3 · p. 3, §3.1 · p. 3, §3.5 · p. 6
3.2 The BrachistoneLR schedule
- BrachistoneLR maps the cycloid’s normalized vertical coordinate between peak and floor rates; algebraically, it is cosine annealing with half-period E−1.It reaches ηmin at the final trained epoch; the terminal-rate difference scales as E^-2 and becomes negligible on long horizons.Source: §3.2 · pp. 4–5
3.3 Baseline schedules
- The five baselines span constant, calendar-based, smooth peak-to-floor, and warmup-plus-smooth policies; exponential decay ends above the floor while step decay ends an order below it.The calendar-based schedules are reported as conventionally defined, without clipping them to the nominal floor.Source: §3.3 · p. 5, Table 1 · p. 5
3.4 Architectures
- The benchmark uses fully connected, convolutional, recurrent, and residual architecture families to test whether schedule behavior generalizes across inductive biases and gradient-flow regimes.The recurrent model is an LSTM, and the residual model uses skip connections.Source: §3.4 · pp. 5–6
3.5 Datasets and training protocol
- BrachistoneLR is best on three dataset–architecture configurations, including both residual networks, ranks among the top three in ten of twelve rows, and is never worse than fourth.Each configuration was run once with a single random seed, so sub-percentage-point differences are treated as indicative rather than statistically significant.Source: §3.5 · p. 6, Table 2 · p. 7
4 Results
Across the benchmark, smoothly declining schedules outperform constant and calendar-based schedules, while BrachistoneLR, cosine annealing, and warmup-cosine are too close for their ordering to be meaningful with one seed. BrachistoneLR’s strongest results are on residual networks and CIFAR-10, and the trajectory comparison highlights its endpoint timing.
4 Results
- The results cover twelve dataset–architecture combinations, with each reported accuracy taken from the epoch of highest validation accuracy in a single run.Small differences are therefore indicative rather than a strict ordering.Source: §4 · p. 7
4.1 Overall comparison
- BrachistoneLR records 83.38% on the CIFAR-10 ResNet, 1.20 points above cosine annealing and the highest single accuracy recorded on CIFAR-10.Across the twelve configurations, BrachistoneLR wins on both residual networks and MNIST with the LSTM.Source: §4.1 · p. 7
4.2 Aggregate behavior
- 86.10%, 86.16%, and 86.13% overall accuracy for cosine annealing, warmup-cosine, and BrachistoneLR, respectively; the three are separated by just 0.06 points and 0.17 of a mean rank.On CIFAR-10, BrachistoneLR has the highest dataset mean at 68.21%, just ahead of warmup-cosine at 68.17% and cosine annealing at 67.90%; one seed per configuration cannot establish the leading group’s ordering.Source: §4.2 · p. 8, Table 3 · p. 8, §4.1 · p. 8
4.3 Architecture and dataset effects
- BrachistoneLR is strongest on ResNets, averaging 92.04% across datasets, but trails exponential decay on the LSTM mean, 81.27% to 81.41%.On convolutional networks it remains within 0.06, 0.30, and 0.58 points of the row leader on MNIST, Fashion-MNIST, and CIFAR-10, respectively.Source: §4.3 · p. 8
4.4 Learning-rate trajectories
- BrachistoneLR and warmup-cosine reach the floor when training stops, while cosine annealing does not; BrachistoneLR alone avoids starting below the peak.The reported per-epoch validation curves showed transient accuracy dips after step-decay drops, unlike the smooth schedules.Source: §4.4 · p. 9
5 Discussion
Smooth peak-to-floor schedules outperform constant and calendar-based policies, with gaps that grow as tasks become harder. The leading smooth schedules are nearly tied, so the practical choice turns on tuning demands and the study’s limited evidence.
5 Discussion
- The smooth schedules finish within 0.06 points of one another, while constant and calendar-based policies trail by half a point to a point overall, with gaps reaching 2.5 dataset-mean points on CIFAR-10.On the CIFAR-10 residual network, BrachistoneLR reaches 83.38% versus 77.60% for a constant rate.Source: §5 · p. 9
- Compared with standard cosine annealing, BrachistoneLR’s terminal-rate advantage depends on training horizon and falls off as E^-2; at E = 100, final rates differ by only a few percent of the floor.Source: §5 · p. 10
- Because accuracy is practically similar within the smooth group, the paper recommends choosing by tuning burden; cosine annealing with its half-period set to the training horizon is presented as a reasonable default.The paper particularly notes residual and convolutional models.Source: §5 · p. 11
6 Conclusion
The benchmark separates smooth peak-to-floor schedules from constant and calendar-based decay, while finding little basis to rank the leading smooth schedules. BrachistoneLR is a cosine-period configuration whose terminal-rate distinction shrinks with horizon, not a new schedule family.
6 Conclusion
- Across 72 runs, smooth peak-to-floor schedules beat constant and calendar-based decay by margins that grow with task difficulty.The comparison used a fixed protocol.Source: §6 · p. 11
- The three leading smooth schedules differ by less than a tenth of an accuracy point, an ordering one seed per configuration cannot establish as real.Source: §6 · p. 11
- BrachistoneLR is best on both residual networks and has the highest mean on CIFAR-10, but the paper does not claim this ordering is real.Source: §6 · p. 11
- BrachistoneLR is cosine annealing with half-period E−1, so its floor arrives at the last trained epoch and its deviation shrinks as E−2.Source: §6 · p. 11
- The authors conclude that choosing a smooth schedule matters more than choosing among smooth schedules, and propose repeated paired tests, horizon and peak-rate sweeps, and horizon-free comparisons.Source: §6 · p. 11