Leveraged Weighted Loss for Partial Label Learning

1. Introduction

• Theoretical guarantees for partial label learning remain uncommon and often rely on strict assumptions, while theory-based guidance for parameter selection remains open.Examples of strict assumptions include convex losses and uniformly sampled partial-label sets.Source: §1 · p. 1
• The practical LW algorithm iteratively identifies weighting parameters.Source: §1 · p. 2
• LW loss introduces leverage parameter β to trade off losses on partial labels and non-partial labels.Source: §1 · p. 2
• The authors generalize the uniform partial-label-generation assumption, prove risk consistency for LW loss, and derive guidance for β from the supervised loss to which LW is risk consistent.They also report proving Bayes consistency.Source: §1 · pp. 1–2
• Experiments verify the theoretical guidance and report LW effectiveness against state-of-the-art methods on benchmark and real datasets.Source: §1 · p. 2

2. Related Works

• Average-based methods weight candidate labels equally, while identification-based methods select one candidate as the truth; both strategies have distinct modeling approaches.Average-based approaches can be distracted by false positives, while identification-based approaches are described as vulnerable to differentiated candidate sets.Source: §2 · p. 2
• Identification-based literature uses likelihood, margin, and DNN techniques, but false positives co-occurring with the true label are identified as a weakness.Source: §2 · p. 2
• The paper draws on the general scheme of constructing multiclass losses by combining multiple binary classification losses.Source: §2 · p. 2
• The paper follows identification-based methods and positions LW risk-consistency results as theoretical insight into parameter-dependent performance.Source: §2 · p. 2

3.1. Preliminaries

• Risk consistency equates partial-label and supervised risks for the same classifier and implies classifier consistency.The classifier learned from partial labels then has the same optimum as one learned from fully supervised data.Source: §3.1 · p. 3

3.2. Leveraged Weighted (LW) Loss Function

• LW loss combines a binary loss ψ, label weights, and leverage parameter β to trade off losses on partial and non-partial labels.The binary loss encourages higher scores for candidate labels and penalizes high scores for labels outside the set.Source: §3.2 · p. 3

3.3. Theoretical Interpretations

• The theoretical analysis generalizes partial-label sampling assumptions, proves risk consistency, and uses the corresponding supervised loss to guide β selection.The paper motivates β guidance by examining which supervised loss the LW loss is risk-consistent with.Source: §3.3 · p. 3, §3.3.3 · p. 4

3.3.1. GENERALIZING THE UNIFORM SAMPLING ASSUMPTION

• The partial-label set may be generated with label-specific inclusion probabilities: q_y = 1, while q_z < 1 for other labels under the small ambiguity degree condition.The paper assumes partial-label elements are independently drawn.Source: §3.3.1 · p. 4

3.3.2. RISK-CONSISTENT LOSS FUNCTION

• LW loss is risk-consistent under label-specific partial-label sampling, and is Bayes-consistent for β > 0 when ψ is differentiable and symmetric.The analysis also applies to deterministic and stochastic scenarios.Source: §3.3.2 · p. 4

3.3.3. GUIDANCE ON THE CHOICE OF β

• β=0 is discouraged; β=1 is reasonable, and β>0—especially β>1—is preferred; β=2 is also preferred when confusing-label sampling is weighted by q_z.The analysis says β controls the relative scale of losses on non-true labels, not the loss on the true label.Source: §3.3.3 · p. 5

3.4. Practical Algorithm

• Because the true label is hidden, the algorithm iteratively learns weights that emphasize it and the highest-scoring non-partial label.It separately normalizes partial and non-partial labels to keep partial-label weights from growing until β and non-partial losses become ineffective.Source: §3.4 · pp. 5–6

4. Experiments

• Experiments assess LW against other methods on benchmark and real datasets and probe parameter choice, loss components, and data-generation effects.The experiments also examine empirical understandings beyond the classification comparisons.Source: §4 · p. 6, §4.1 · p. 6, §4.2 · p. 8

4.1. The Classification Performance

• LW outperforms almost all state-of-the-art algorithms with Sigmoid loss on benchmarks and is best on almost all real datasets under the same base model.The benchmark results also report significant cross-entropy improvements on MNIST, Fashion-MNIST, and Kuzushiji-MNIST.Source: §4.1.1 · p. 6, Table 1 · p. 7, §4.1.2 · p. 7

4.1.1. BENCHMARK DATASET COMPARISONS

• With Sigmoid loss, LW outperforms almost all state-of-the-art algorithms on benchmarks; cross-entropy LW significantly improves results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST.The benchmark experiments use four datasets and generate partial labels with q ∈ {0.1, 0.3, 0.5}, retaining the true label in every set.Source: §4.1.1 · p. 6, Table 1 · p. 7

4.1.2. REAL DATA COMPARISONS

• Under the same base model, LW achieves the best performance on almost all real datasets.The paper describes the real partial-label distribution as unknown and potentially more complex than the benchmark generation process.Source: §4.1.2 · p. 7, Table 2 · p. 7

4.2. Empirical Understandings

• The empirical-understandings experiments examine LW's effectiveness through parameter analysis, an ablation study, and alternative partial-label generation..Source: §4.2 · p. 8

4.2.1. PARAMETER ANALYSIS

• On all four datasets across varying q, β = 1 and β = 2 significantly outperform other settings; on MNIST, β = 4 is also competitive.These outcomes coincide with the paper's theoretical guidance for choosing β.Source: Figure 1 · p. 8, §4.2.1 · p. 8

4.2.2. ABLATION STUDY

• Using losses on partial labels alone or non-partial labels alone yields unsatisfactory accuracy on all three datasets, while combining both with α = β = 1 performs better.The combined loss is especially effective on CIFAR-10 and Kuzushiji-MNIST; simply scaling the losses provides little help.Source: Figure 2 · p. 8, §4.2.2 · p. 8

4.2.3. THE INFLUENCE OF DATA GENERATION

• LW dominates its counterparts in all three alternative data-generation cases; on Kuzushiji-MNIST, its accuracy advantage over PRODEN rises from 0.71% in Case 1 to 1.25% in Case 3.Accuracy drops for all methods as the generation process becomes more complex, but LW-Cross entropy shows stronger resistance.Source: §4.2.3 · pp. 8–9, Table 3 · p. 9

5. Conclusion

• LW loss addresses partial-label learning, and the paper's theoretical and empirical analyses favor β = 1 and β = 2.The conclusion reports that comparisons on benchmark and real datasets demonstrate the method's effectiveness.Source: §5 · p. 9

Appendix

• The supplement contains the proof of Theorem 1 and detailed numerical-experiment settings, including datasets, compared methods, model architectures, and data generation.Source: Appendix · p. 11

• Theorem 3 gives a risk-consistency result for a partial loss under the generalized label-specific sampling assumption.The theorem identifies a supervised loss with respect to which the partial loss is risk-consistent.Source: A. Proofs · p. 11
• When the stated condition holds, the partial risk equals the supervised risk, as expressed by R̄(L̄, g(X)) = R(L, g(X)).The proof notes that partial-label sets excluding the true label have probability zero under the specified sampling setup.Source: A. Proofs · p. 11
• For β > 0, optimizing LW and the constraint comparison method yields the same classifier; with differentiable ψ, the cited result implies Bayes classification consistency.The proof invokes Example 3 of Tewari & Bartlett (2007) for the constraint-comparison result.Source: A. Proofs · p. 13

• The benchmarks are MNIST, Fashion-MNIST, Kuzushiji-MNIST, and CIFAR-10, each a 10-class image dataset.The first three use 28 × 28 grayscale images; CIFAR-10 uses 32 × 32 × 3 RGB images.Source: §B.1.1 · p. 13

• The five real-world datasets are Lost, BirdSong, MSRCv2, Soccer Player, and Yahoo! News.Source: §B.1.2 · p. 13
• Lost, Soccer Player, and Yahoo! News use faces in images or video frames as instances and caption or subtitle names as candidate labels.Source: §B.1.2 · p. 13
• MSRCv2 uses image segments and co-occurring objects as samples and candidate labels, while BirdSong uses singing syllables and co-singing bird species.Source: §B.1.2 · p. 13
• Table 5 reports the average number of candidate labels per instance.Source: §B.1.2 · p. 13, Table 5 · p. 14

B.2. Compared Methods

• IPAL propagates label confidence iteratively, PALOC uses one-vs-one decomposition, and PLECOC transforms the task into a binary-label problem with E-COC coding.Source: §B.2 · p. 14
• PRODEN combines a risk estimator with progressive identification, while RC is risk-consistent and CC is classifier-consistent based on a generation model.Source: §B.2 · p. 14
• The reported shared setup uses cross-entropy, a 10% training-set validation sample, batch size 256, and 250 epochs.Source: §B.2 · p. 14

B.3. Details of Architecture

• The linear model maps input dimension d to 10 outputs; the five-layer MLP uses four 300-unit hidden layers, ReLU, batch normalization, softmax, and L2 regularization.Source: §B.3 · p. 14
• The ConvNet uses repeated 128- and 256-channel 3×3 convolution blocks with max pooling, followed by 512-, 256-, and 128-channel convolutions and average pooling to 10 outputs.Source: §B.3 · p. 14

• Case 1 assigns each true label one unique similar label with probability q1; at q1 = 0.5, it matches the generation proposed by Lv et al. (2020).Source: §B.4 · pp. 14–15
• The matrix entries represent conditional probabilities P(j ∈ Y | Y = i, x) for a label to enter the partial-label set given the true label and instance.Source: §B.4 · p. 15
• Case 2 gives each true label two similar labels, each entering with q1 = 0.3.Source: §B.4 · p. 15
• Case 3 uses one pair of most-similar labels at q1 = 0.5 and two less-similar pairs at q2 = 0.3 and q3 = 0.1.Source: §B.4 · p. 15