PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions

1 INTRODUCTION

• Loss design is challenging because a function that works for one task may not work for another; for example, L1/L2 losses are common for regression but rare for classification.Focal loss was proposed to alleviate cross-entropy overfitting on imbalanced detection datasets, but is not shown to help consistently elsewhere (Lin et al., 2017).Source: §1 · p. 1
• PolyLoss represents classification losses as weighted bases (1 − Pt)^j, with coefficients αj that can be adjusted for different tasks and datasets.Setting αj = 1/j recovers cross-entropy; the framework also explains cross-entropy and focal loss as special cases, while allowing vertical coefficient adjustments beyond focal loss’s horizontal shifts.Source: §1 · pp. 1–2, §2 · p. 3
• Adjusting only the leading polynomial coefficient, called LPoly-1, can significantly improve results over cross-entropy and focal loss.The proposed Poly-1 formulation uses one extra hyperparameter and one line of code; experiments report improvements across classification, segmentation, and detection tasks.Source: §1 · p. 2
• The authors report that focal loss is suboptimal on imbalanced ImageNet-21K, and that the leading polynomial contributes substantially to training gradients and correlates with Pt.They use this correlation to motivate tailoring losses to imbalanced datasets; their experiments report consistent improvements across tasks, models, and datasets.Source: §1 · p. 2

2 RELATED WORK

• Cross-entropy appears in state-of-the-art perception models, and prior work has proposed alternatives; this paper instead aims to provide a unified framework for systematically designing classification losses.The passage cites Tan & Le (2021), He et al. (2017), Zoph et al. (2020), and Tao et al. (2020) for cross-entropy use, and Lin et al. (2017), Law & Deng (2018), Cui et al. (2019), and Zhao et al. (2021) for alternative losses.Source: §2 · p. 2
• Focal loss addresses class imbalance by focusing on hard examples and is used in 2D and 3D detectors, but the authors find it suboptimal for imbalanced ImageNet-21K.PolyLoss yields a loss that plays the opposite role of focal loss, supporting the authors’ argument for tailoring losses to different imbalanced datasets. Focal loss is cited as Lin et al. (2017); detector examples are Tan et al. (2020), Du et al. (2020), Shi et al. (2020), and Sun et al. (2021).Source: §2 · p. 2
• Taylor cross entropy unifies MAE and cross-entropy through (1 − Pt)^j bases, and truncating higher-order terms makes it more robust to synthetic label noise.PolyLoss generalizes coefficient manipulation to include dropping higher-order terms, while the authors report Feng et al. (2020)’s loss performs worse than cross-entropy on clean ImageNet.Source: §2 · p. 2
• Learned-loss approaches include gradient-based and meta-learning methods; TaylorGLO uses CMA-ES to optimize a multivariate Taylor loss and learning-rate schedule (Hansen & Ostermeier, 1996; Gonzalez & Miikkulainen, 2020b).The authors state that hundreds of polynomials are needed for 1000-class tasks, making the search space prohibitively large; Poly-1 instead uses a one-parameter grid search and avoids advanced black-box optimization.Source: §2 · pp. 2–3

Cross-entropy loss as PolyLoss

• In the cross-entropy expansion, coefficients 1/j cancel the jth power upon differentiation, yielding a gradient sum whose leading term is constant and whose higher-order terms are suppressed as Pt approaches 1.Source: §3 · p. 3

Focal loss as PolyLoss

• Focal loss shifts polynomial powers by γ; for positive γ, its gradient drops cross-entropy’s constant leading term, suppressing the first term as (1 − Pt)^γ to reduce emphasis on confident majority-class examples.Source: §3 · pp. 3–4

• Poly-1 is the final loss formulation, simplifying Poly-N to a grid search over one hyperparameter, while Poly-N perturbs the leading N coefficients and dropping higher-order terms truncates the expansion.Source: Table 2 · p. 4

Connection to regression and general form

• When y = 1 represents the ground-truth label, the bases can be written as (y − Pt)^j, making cross-entropy and focal loss weighted ensembles of powered prediction-to-label distances and raising whether their coefficients are optimal.Source: §3 · p. 4
• PolyLoss is a monotone decreasing function on [0, 1], expressed as a weighted sum of (1 − Pt)^j with adjustable coefficients; although non-integer powers are possible, this paper focuses on positive integer j.Source: §3 · p. 4

4 UNDERSTANDING THE EFFECT OF POLYNOMIAL COEFFICIENTS

• The section compares dropping higher-order terms, adjusting multiple leading coefficients, and adjusting only the first coefficient, finding the last offers the greatest gain with minimal tuning.Experiments use 1000-class ImageNet-1K and ResNet-50 with its original training hyperparameters.Source: §4 · p. 4

4.1 LDrop: REVISITING DROPPING HIGHER-ORDER POLYNOMIAL TERMS

• More than 600 polynomial terms are needed to match cross-entropy accuracy, and changing the learning rate does not remedy the accuracy loss from truncation.The theorem and experiments indicate that high-order terms matter early in training, when Pt is small; at Pt ∼0.001, the 500th term’s gradient coefficient is about 0.6.Source: §4.1 · pp. 4–5, Figure 2 · p. 5, §7 · p. 11

4.2 LPOLY-N: PERTURBING LEADING POLYNOMIAL COEFFICIENTS

• Poly-N perturbs the first N cross-entropy coefficients while leaving higher terms unchanged; N=1 improves accuracy by +0.4, while N=2 adds only +0.1.Searching across many coefficients creates a prohibitively large space, and collectively tuning them does not outperform cross-entropy.Source: §4.2 · pp. 5–6, Table 3 · p. 5

4.3 LPOLY-1: SIMPLE AND EFFECTIVE

• Increasing Poly-1’s first coefficient systematically improves ResNet-50 accuracy over cross-entropy; the added term requires only one line of code.The improvement holds even though training hyperparameters were optimized for cross-entropy; the first polynomial contributes more than half of the cross-entropy gradient for most training steps.Source: Figure 3 · p. 6, §4.3 · p. 6

5 EXPERIMENTAL RESULTS

• Poly-1 gains around 1% accuracy across EfficientNetV2 sizes on ImageNet-21K pretraining, improves Mask R-CNN AP and AR on COCO, and significantly improves all reported PointPillars metrics on Waymo.These results use Figure 4 and Tables 5 and 7; the optimal coefficient varies across tasks.Source: §5.1 · p. 7, Figure 4 · p. 7, §5.2 · p. 8

5.1 LPOLY-1 IMPROVES 2D IMAGE CLASSIFICATION ON IMAGENET

• On imbalanced ImageNet-21K, increasing the leading coefficient improves pretraining accuracy (+0.6), while reducing it lowers accuracy (-0.9); positive ϵ1 also raises mean Pt.The authors hypothesize that positive ϵ1 helps because ImageNet-21K predictions are initially insufficiently confident.Source: §5.1 · p. 7
• 87.2% top-1 accuracy on ImageNet-1K, up from 86.8% with cross-entropy, is achieved by finetuning the ImageNet-21K-pretrained EfficientNetV2-L with Poly-1.Table 4 reports ImageNet validation classification accuracy; both losses use ϵ1 = 2.Source: §5.1 · p. 7, Table 4 · p. 7

5.3 LPOLY-1 IMPROVES 3D OBJECT DETECTION ON WAYMO OPEN DATASET

• On Waymo, Poly-1 with ϵ = −1 significantly improves all reported metrics for PointPillars vehicle and pedestrian detection.The method replaces the classification focal loss in the single-stage detector.Source: §5.3 · p. 9, Table 7 · p. 9
• For the two-stage RSN detector, tuning the new leading polynomial improves all reported metrics except vehicle BEV APH L2.RSN’s new leading polynomial is (1 − Pt)^4.Source: §5.3 · p. 9, Table 7 · p. 9

6 CONCLUSION

• PolyLoss provides a unified view of common classification losses by representing loss functions through polynomial coefficients.The framework enables flexible changes to loss-function shape by adjusting those coefficients.Source: §6 · p. 9
• Under polynomial expansion, focal loss is a horizontal shift of cross-entropy loss coefficients, motivating exploration of vertical coefficient modifications.Source: §6 · p. 9
• Poly-1 adjusts the leading polynomial coefficient with one extra hyperparameter, ϵ1, and improves models across multiple tasks and datasets.The authors emphasize its simplicity as requiring one extra line of code.Source: §6 · p. 9
• The authors argue that simple loss-function modifications can improve established state-of-the-art models and encourage exploration beyond cross-entropy, focal loss, and Poly-1.Source: §6 · p. 9

• The experiments use public datasets and open-source code, leave default training hyperparameters unchanged, and modify only loss functions; final LPoly-1 requires one code-line change.Source: REPRODUCIBILITY STATEMENT · p. 10
• For softmax, Poly-1 adds epsilon * (1 - pt) to cross-entropy, where pt is the summed label-weighted softmax probability.Source: REPRODUCIBILITY STATEMENT · p. 10
• With label smoothing, Poly-1 adds epsilon times one_minus_pt to the smoothed-label cross-entropy.Source: REPRODUCIBILITY STATEMENT · p. 10
• For sigmoid activation, Poly-1 focal loss adds epsilon * (1 - pt)^(gamma + 1) to focal loss, with pt computed from labels and sigmoid probabilities.Source: REPRODUCIBILITY STATEMENT · p. 10
• The alpha-balanced sigmoid variant weights the added epsilon term by labels * alpha + (1 - labels) * (1 - alpha).Source: REPRODUCIBILITY STATEMENT · p. 10

7 PROOF OF THEOREM 1

• For ζ > 0 and δ > 0, N > log1−δ(ζ · δ) implies |R_N(p)| < ζ for every p ∈ [δ, 1].The supplied theorem statement is truncated after beginning a further bound on |R′.Source: §7 · p. 11

8 ADJUSTING OTHER TRAINING HYPERPARAMETERS LEADS TO HIGHER GAIN.

• PolyLoss gained +0.8% after adjusting ResNet50 weight decay, exceeding the best cross-entropy accuracy of 76.3% by +0.8%.The default 1e-4 weight decay was optimized for cross-entropy; adjusting it could reduce cross-entropy performance while producing a larger gain for PolyLoss.Source: Table 8 · p. 11, §8 · p. 11
• Tuning the leading polynomial term did not improve COCO detection AP when hyperparameters were optimized for the optimal Focal loss values; the authors suspect a local maximum.The ablation adjusted γ alone or both γ and α.Source: §8 · p. 11
• On COCO RetinaNet, adjusting only γ or both γ and α let PolyLoss consistently outperform the best Focal Loss AP of 33.4.The optimal Focal loss balance values were (γ, α) = (2.0, 0.25).Source: Table 9 · p. 11, §8 · p. 11

9 LDROP WITH MORE HYPERPARAMETER TUNING

• For LDrop (N = 2), the tuning also adjusts learning rate and weight decay alongside the second polynomial’s coefficient α, similar to Gonzalez & Miikkulainen (2020b).Source: §9 · p. 11
• Unlike Feng et al. (2020), who set α = 0.5 after dropping all higher-order polynomial terms, the optimal α is 8 at default learning rate 0.1, increasing accuracy to 70.9.The authors conclude that simply dropping polynomial terms is insufficient and adjusting polynomial coefficients is critical.Source: §9 · pp. 11–12
• Further weight-decay tuning yields less than 0.1% model-quality improvement for LDrop.Source: §9 · p. 12
• Poly-1 outperforms LDrop*; tuning its weight decay further increases accuracy while requiring fewer hyperparameters (Table 10).Source: Table 10 · p. 12, §9 · p. 12

10 COLLECTIVELY TUNING MULTIPLE POLYNOMIAL COEFFICIENTS

• Collective tuning replaces cross-entropy’s 1/j coefficients with exponential decay and truncates the infinite sum at twice the decay factor N.Source: §10 · p. 12
• 72.3 is the best accuracy, at N = 80 and learning rate 1.6, in a grid search over N ∈ {5, 20, 80, 320} and learning rates {0.1, 0.4, 1.6, 6.4} (Table 11).Source: Table 11 · p. 12, §10 · p. 12
• Poly-1 performs better than Lexp, but other collective coefficient schemes remain possible and their effects on training warrant further study.Source: §10 · p. 12

11 COMPARING TO OTHER TRAINING TECHNIQUES

• Poly-1 achieves improvements similar to label smoothing and dropout on FC for ResNet50 ImageNet-1K accuracy.Table 12 reports the comparison, but the supplied passages do not provide its cell values.Source: §11 · p. 12, Table 12 · p. 12
• Prior work reports that independent novel training techniques often improve results by under 1%, while combining techniques can yield significant overall gains (He et al., 2019; Bello et al., 2021; Wightman et al., 2021).This motivates comparing Poly-1 with commonly used training techniques.Source: §11 · p. 12

12 REDISCOVERING FOCAL LOSS FROM POLYLOSS

• Focal loss was developed for RetinaNet to address the strong class imbalance in object detection (Lin et al., 2017).The section presents its PolyLoss analysis as an additional ablation study.Source: §12 · p. 13
• The study systematically rediscovers focal loss by starting from cross-entropy’s PolyLoss representation and examining its gradients.It investigates how leading terms affect training under class imbalance.Source: §12 · p. 13
• Dropping the first two cross-entropy polynomial terms significantly improves RetinaNet detection AP and AR and gives its best performance, similar to focal loss with γ = 2.The paper relates focal loss with γ = 2 to shifting polynomial coefficients right by two, similar to truncating the first two terms.Source: §12 · p. 13, Figure 8 · p. 13
• The leading cross-entropy polynomial is constant, and its binary-classification gradient is Nbackground−Nobject, so extreme class imbalance lets the majority class dominate optimization.This bias motivates dropping leading polynomials to reduce overfitting to the majority class.Source: §12 · p. 13
• Dropping polynomials reduces extremely confident overall predictions, lowers background confidence, and increases object confidence by reducing overfitting to the majority background class.Overall confidence is dominated by background confidence because of the extreme class imbalance.Source: §12 · p. 13