PaperlayerView exact revision on arXiv

Source-linked AI summary

IT-OSE: Exploring Optimal Sample Size for Industrial Data Augmentation

Mingchun Sun, Rongqiang Zhao, Zhennan Huang, Songyu Ding, Jie Liu

arXiv:2602.15878v1cs.LGcs.AI

TL;DR

Industrial augmentation lacks a reliable theoretical way to choose optimal sample size and evaluate its deviation from ground truth. The paper introduces IT-OSE and ICD, reporting better downstream performance and determinacy than empirical estimation while matching exhaustive-search OSS at substantially lower costs.

  • Problem

    Industrial data augmentation has no established theoretical OSS estimator or intuitive metric for evaluating OSS accuracy and deviation from ground truth.

  • Method

    IT-OSE theoretically relates OSS to dominant dataset, baseline-model, and generator factors, while ICD evaluates interval coverage and deviation.

  • Results

    IT-OSE improved classification accuracy by 4.38%, reduced regression MAPE by 18.80%, and reduced ICDdev by 49.30% versus empirical estimation; versus exhaustive search, it reduced computational and data costs by 83.97% and 93.46%.

  • Takeaways & Limitations

    IT-OSE achieved the same OSS as exhaustive search with lower computational and data costs and showed generality across representative sensor-based industrial scenarios.

Abstract

from arXiv · show

In industrial scenarios, data augmentation is an effective approach to improve model performance. However, its benefits are not unidirectionally beneficial. There is no theoretical research or established estimation for the optimal sample size (OSS) in augmentation, nor is there an established metric to evaluate the accuracy of OSS or its deviation from the ground truth. To address these issues, we propose an information-theoretic optimal sample size estimation (IT-OSE) to provide reliable OSS estimation for industrial data augmentation. An interval coverage and deviation (ICD) score is proposed to evaluate the estimated OSS intuitively. The relationship between OSS and dominant factors is theoretically analyzed and formulated, thereby enhancing the interpretability. Experiments show that, compared to empirical estimation, the IT-OSE increases accuracy in classification tasks across baseline models by an average of 4.38%, and reduces MAPE in regression tasks across baseline models by an average of 18.80%. The improvements in downstream model performance are more stable. ICDdev in the ICD score is also reduced by an average of 49.30%. The determinism of OSS is enhanced. Compared to exhaustive search, the IT-OSE achieves the same OSS while reducing computational and data costs by an average of 83.97% and 93.46%. Furthermore, practicality experiments demonstrate that the IT-OSE exhibits generality across representative sensor-based industrial scenarios.

I. INTRODUCTION

Industrial data augmentation can improve robustness and generalization, but its sample size has competing effects: too little limits learned variation, while too much can introduce redundancy, shifts, noise, or overfitting. The paper proposes IT-OSE and ICD to estimate and evaluate OSS more theoretically and efficiently.

  • Insufficient augmentation limits the model’s ability to capture task-relevant input variation, whereas excessive augmentation can cause redundancy, distributional shifts, label noise, or overfitting.
  • Exhaustive search determines OSS from performance curves but requires repeated experiments, while empirical estimation is faster but relies on domain knowledge.
  • Existing approaches lack clear theoretical determinants, reliable OSS estimation, intuitive deviation metrics, and broad generality across sensor-based industrial scenarios.
  • The paper proposes IT-OSE to identify dominant factors and estimate OSS without exhaustive search or domain-specific experience, alongside the ICD score for intuitive evaluation.
  • IT-OSE theoretically relates OSS to the dataset, baseline model, and generator, while experiments report improved determinacy and reduced computational and data costs.

II. METHODOLOGY

The IT-OSE covers classification and regression augmentation through separate estimators matched to whether augmentation expands information content.

  • ITLE estimates OSS for extended augmentation, which is mainly used for classification when class-wise samples support generative-model training.
  • MGEE estimates OSS for non-extended augmentation, which is mainly used for regression tasks with limited samples per label.
  • Together, ITLE and MGEE cover classification and regression tasks in industrial scenarios.

A. ITLE for Extended Data Augmentation

ITLE estimates the augmentation needed to close an information gap between a dataset and baseline model, using mutual information, model error, complexity, and conservative bounds. It reports OSS as a constrained interval based on dataset, model, and generator factors.

  • ITLE pipeline: ITLE estimates OSS for extended augmentation by filling the information gap between the given dataset and baseline model.
  • ITLE pipeline: The procedure estimates mutual information from training data while separating test data to prevent information leakage.
  • ITLE pipeline: The baseline model’s test error is compared with an information-theoretic lower reference to quantify the model’s information gap.
  • ITLE pipeline: Rademacher complexity quantifies model-family expressive capacity and supports estimation of the sample size needed to fill the information gap.
  • ITLE pipeline: The target tolerance determines a generalization-error saturation threshold, while correction factors account for bound looseness and model or loss-function effects.
  • ITLE pipeline: ITLE can return zero augmentation when the model is saturated on the current dataset, even when the measured information gap is nonzero.
  • ITLE pipeline: The OSS estimate is adjusted into a conservative interval using dataset, baseline-model, and generator factors, with information contribution quantified between real and augmented data.

B. MGEE for non-extended data augmentation

MGEE estimates OSS for non-extended augmentation by modeling generalization error as augmentation increases. It uses dataset, baseline-model, and generator factors to identify where performance saturates and returns an OSS interval.

  • Non-extended augmentation preserves informational support, so MGEE estimates OSS by mitigating the baseline model’s generalization error.
  • MGEE trains the baseline model, estimates effective model complexity with PAC-Bayes theory, and computes theoretical generalization error G(a).
  • The estimation uses training, validation, and test splits to support model selection and prevent information leakage.
  • The effective sample size accounts for information redundancy between augmented and original data, which depends on the generator.
  • The optimal augmentation ratio a* is the point where G(a) stops decreasing as augmentation increases, indicating performance saturation.
  • MGEE returns an OSS interval whose bounds are corrected for model complexity, redundant augmented data, and empirical error fluctuations.

C. ICD Score for Evaluation

The ICD score evaluates estimated OSS against ground truth using interval coverage and scale-normalized deviation. Its two components capture whether the interval covers the reference and how far its midpoint deviates.

  • The ICD score combines qualitative interval coverage with quantitative deviation from the ground-truth OSS.
  • The reference quantile Q(N) normalizes sample-size magnitude differences, enabling comparisons across datasets with different scales.
  • ICDcov equals 1 when the estimated OSS interval contains the reference OSS and 0 otherwise.
  • ICDdev measures deviation between the estimated interval midpoint and the reference OSS, using the midpoint because OSS has lower and upper bounds.
  • The final ICD score integrates ICDcov and ICDdev into a comprehensive evaluation of IT-OSE estimates.

A. Industrial Scenarios and Experimental Setup

Experiments cover two sensor-based industrial scenarios, classification and regression datasets, diverse baseline models, and extended and non-extended augmentation methods. The setup compares generated signals with real signals while preserving semantic consistency.

  • Industrial scenarios: Experiments use rice-quality detection with olfactory sensing and potassium-ion measurement with electrochemical sensing.
  • Datasets: The classification dataset contains 297 balanced samples across normal, expired, and moldy rice categories.
  • Datasets: The regression dataset contains 79 samples, including 70 training samples and 9 test samples for concentration measurement.
  • Models: Models span logistic regression, SVM, ANN, CNN, LSTM, and Transformer architectures with varying complexity.
  • Augmentation: Extended augmentation uses ImageTime, Diffusion-TS, and DiffTime, while non-extended augmentation uses RIM, TSW, DTW, and Non-SIM.
  • Signal comparison: Generated signals provide additional information while maintaining semantic consistency rather than duplicating real signals.

B. Exploring OSS with IT-OSE

The experiments estimate OSS with ITLE and MGEE, validate interval reliability, and compare IT-OSE with empirical estimation and exhaustive search. IT-OSE improves downstream metrics, reduces ICD deviation, and lowers computational and data costs while attaining the same OSS as exhaustive search.

  • OSS and key parameters: ITLE and MGEE estimate OSS for classification and regression tasks using fixed datasets, baseline models, and grid-searched hyperparameters.The experiments use K=5, B=200, §=0.05, and v=0.03 for stability.
  • Validation experiments: The shaded OSS intervals largely cover the ground truth, supporting the reliability of ITLE and MGEE estimates.Validation uses repeated experiments across sample sizes from 100 to 700.
  • Validation experiments: Excessive augmentation can underfit low-capacity models when added information becomes too large relative to the given dataset.The LR example attributes this behavior to insufficient model capacity.
  • Evaluation with ICD scores: 49.30% lower ICDdev accompanies IT-OSE estimates, while increased ICD coverage and metric stability indicate more determinate OSS estimation.The comparison is against empirical baselines across the reported industrial tasks.
  • Comparisons with existing approaches: 4.38% higher average classification accuracy and 18.80% lower average regression MAPE are reported for IT-OSE versus empirical estimation.These averages are reported across baseline models in the respective tasks.
  • Comparisons with existing approaches: 83.97% lower computational cost and 93.46% lower data cost are achieved while retaining the same OSS as exhaustive search.The comparisons use computational time T' and generated sample size Size.

C. Investigation of dominant factors’s Influence

The paper investigates how dataset size and baseline-model complexity influence OSS. Larger datasets and more complex models are associated with larger OSS under the examined settings.

  • Given Dataset: Increasing the given dataset size increases usable information and the OSS required to fill the remaining information gap.The analysis randomly retains 90% of each training set for comparison.
  • Baseline Model: Greater baseline-model complexity increases extractable information and correspondingly raises OSS.Low-complexity machine-learning models produced OSS of 0 on the given datasets, while higher-complexity models were used for investigation.

| 6- | RM TSW DTW Non-SIM

Practicality experiments test IT-OSE across representative sensor-based industrial scenarios. IT-OSE generalizes across the scenarios, stabilizes downstream improvements, and matches exhaustive-search performance while outperforming empirical estimation.

  • Practicality Experiments in Industrial Scenarios: Practicality experiments evaluate rice batches and wastewater or tap-water samples using the best offline baseline models and professional-testing ground truth.The comparison includes no augmentation, IT-OSE, exhaustive search, and a 3:2 empirical ratio.
  • Practicality Experiments in Industrial Scenarios: IT-OSE exhibits generality across the examined industrial scenarios and makes downstream performance improvements more stable.The reported comparisons include both IT-OSE and exhaustive-search augmentation.
  • Practicality Experiments in Industrial Scenarios: IT-OSE and exhaustive search achieve the same performances, while IT-OSE outperforms empirical estimation.This supports replacing extensive exploratory experiments in the tested scenarios.

IV. CONCLUSION

The IT-OSE and ICD score provide theoretically derived OSS estimation and intuitive evaluation for industrial data augmentation, with improved interpretability. Results indicate more stable downstream performance, reduced costs, and generality across representative sensor-based industrial scenarios, while future work targets broader factors and applications.

  • IT-OSE uses ITLE and MGEE to provide reliable OSS estimation for extended and non-extended augmentation, respectively.
  • The ICD score provides an intuitive evaluation of estimated OSS, while theoretical analysis formulates relationships between OSS and dominant factors.
  • IT-OSE achieves the same OSS while reducing computational and data costs, with more stable downstream model-performance improvements.
  • Practicality experiments demonstrate IT-OSE generality across representative sensor-based industrial scenarios.
  • Future research will analyze additional OSS factors, narrow the OSS interval, and extend IT-OSE to further industrial scenarios.
  • IT-OSE could serve as a foundation for OSS estimation in industrial data augmentation research.
Loading 2602.15878v1…