Source-linked AI summary
Candidate-Fate Accounting for Transparent Sensor Diagnostic Pipeline Search
Haotao Xie, Yutian Chen, Yangqi Liu, Xiaoyu Jiang
TL;DR
Fitted-trial-only reports omit invalid, pruned, skipped, cached, and unfitted candidates from automated diagnostic search traces. Candidate-fate accounting audits observed candidates, and records about 30, 41, and 34 non-full-evaluated candidates across CWRU, Ottawa, and SEU.
Problem
Existing automated diagnostic search reports typically retain fitted trials, scores, and winners while omitting generated candidates and limiting trace review.
Method
Candidate-fate accounting audits observed diagnostic candidates using hashes, typed legality checks, allocation rationales, and a closed terminal-fate ledger.
Results
About 30, 41, and 34 non-full-evaluated canonical candidates were recorded on CWRU, Ottawa, and SEU, with complete non-overlapping fate partitions.
Takeaways & Limitations
The framework turns rejected, skipped, cached, and unfitted candidates into reviewable evidence while retaining useful final-test performance under a controlled protocol.
Takeaways & Limitations
Empirical validation across domains remains future work.
Abstract
from arXiv · showhide
Industrial sensor diagnostics relies on preprocessing, representation, and classification pipelines, making automated pipeline search useful for reducing manual design cost. However, existing automated machine/deep learning (AutoML/AutoDL) reports typically retain only fitted trials, scores, and winners, omitting generated candidates that are invalid, pruned, skipped, cached, or unfitted. This omission limits reviewers' ability to check signal constraints, budget use, and unevaluated legal alternatives. To address this, we propose candidate-fate accounting, a candidate-level audit framework for diagnostic search traces. It records each observed candidate as auditable evidence: hashes merge repeated observations, legality checks flag invalid candidates, allocation rationales explain budget decisions, and a closed fate ledger assigns one terminal fate to each candidate. Experiments on three bearing-diagnostic datasets show that the framework detects invalid candidates and identifies 30--41 candidates omitted by fitted-trial-only reports, with closed fate records verifying complete candidate accounting while maintaining competitive diagnostic performance. The code is available at https://github.com/XXIE999/candidate-fate-accounting.
1 Introduction
Automated diagnostic-search reports often omit generated candidates that were invalid, budget-blocked, skipped, or otherwise unfitted, creating an audit gap. Candidate-fate accounting addresses this gap by recording observed candidates, their legality, allocation rationale, terminal fate, and closure evidence.
- Related work: Existing AutoML, trial-state, grammar-validity, and provenance tools address parts of candidate auditing but do not close the generated-candidate gap.Their reporting units focus on fitted configurations, scheduled-trial states, illegal programs, or executed artifacts.
- Motivation: Fitted-trial-only reports hide whether generated diagnostic candidates were type-invalid, legal but cost-blocked, or otherwise never fitted.The motivating example contrasts three generated pipelines, of which only one is fitted and reported.
- Framework: Candidate-fate accounting closes records over observed canonical candidates using stable hashes, typed legality checks, allocation rationales, and a terminal fate ledger.The framework merges repeated observations, exposes failures before fitting, explains budget decisions for legal candidates, and verifies closure with Δclose.
- Evaluation: The framework is evaluated on the CWRU, Ottawa, and SEU bearing datasets through invalid-candidate probes, closed fate ledgers, allocation checks, and controlled diagnostic-performance tests.The experiments assess pre-fitting invalidity detection, complete fate accounting, and useful performance under a controlled protocol.
- Contributions: Non-fitted candidates are treated as evidence about signal validity, budget use, and untested legal alternatives rather than discarded search traces.This reframes candidate accounting as an audit requirement for automated industrial diagnostic search.
2 Method
The method treats candidate-fate accounting as an optimizer-independent reporting contract over diagnostic search traces. It combines typed legality checks, reviewable allocation rationale, canonical-hash consolidation, and a closed terminal-fate ledger to account for fitted and non-fitted candidates.
- 2.1 Legal diagnostic candidates: Generated candidates are diagnostic programs comprising preprocessing, representation, and estimator components with declared types and metadata.Type compatibility is checked before fitting, making signal constraints pre-fit evidence.
- 2.1 Legal diagnostic candidates: Legal candidate space is separated from budget admissibility: incompatibilities are invalid before fitting, while cost-, cache-, or termination-blocked legal candidates receive non-evaluated fates.This distinction preserves legality independently of the search budget.
- 2.2 Budget allocation: At each search step, fitting budget is allocated only to generated, legal, uncached candidates without terminal fates.Allocation rationale makes budget use reviewable without changing candidate legality.
- 2.2 Budget allocation: Search policies alter only the priority of legal candidates, not legality, budget eligibility, cache handling, or terminal-fate assignment.Optional label-free descriptors can adjust primitive-family priority but cannot bypass legality or ledger rules.
- 2.3 Terminal fates and closure: The ledger uses canonical SHA-256 hashes to merge repeated observations and assigns each observed candidate one terminal fate, including residual legal-unfitted outcomes.Fixed precedence makes later evidence decisive, while closure verifies that terminal fates form a mutually exclusive and exhaustive partition.
3 Experiments
Experiments test legality, candidate-fate closure, allocation behavior, and diagnostic utility under a controlled small-budget protocol across three bearing-diagnostic datasets. The audit exposes invalid and unevaluated candidates while preserving competitive, protocol-scoped diagnostic performance.
- Protocol: At most B=50 full model-fitting attempts were allowed per dataset and seed, while generated candidates could be skipped, pruned, cached, or recorded as legal-unfitted.All main search conditions shared the typed legal space, primitive cost model, split, and macro-F1 metric.
- Datasets: Three bearing-diagnostic datasets were evaluated: CWRU and Ottawa used approximately 60/20/20 train/validation/test splits, while SEU used the cross-condition split 30_2 →20_0.For SEU, the source condition trained the model and the target condition was divided equally between validation and testing.
- RQ1: Typed legality: 68/100 weakly constrained skeletons were invalid: 48 type-invalid and 20 semantic-invalid, while 32 were legal.The typed main searches were expected to contain zero invalid candidates, and the ledger confirmed invalid candidates did not consume fitting budget.
- RQ2: Candidate-fate closure: 30, 41, and 34 non-full-evaluated canonical candidates were recorded on CWRU, Ottawa, and SEU, respectively, and all GLS ledger rows satisfied Δclose = 0.The closure check verified that fate counts formed a complete, non-overlapping partition of observed canonical candidates.
- RQ3: Allocation behavior: GLS reached the random-search target in 3/3 seeds on every dataset, required the fewest Evaluations to Target among 3/3-success rows, and achieved the best budget-30 F1 on Ottawa and SEU.GLS tied TPE to four decimals on CWRU; the comparison fixed the legal space and budget and was framed as an auditability check rather than broad optimizer ranking.
- RQ4: Diagnostic utility: Among matched search policies, TPE had the highest final-test macro-F1 on CWRU, while GLS was highest on Ottawa and SEU.The controlled-protocol report records selected family, transform, estimator, and seed, supporting auditability alongside diagnostic utility.
4 Conclusion
Candidate-fate accounting closes the generated-candidate audit gap in automated diagnostic search by making rejected, skipped, cached, and unfitted candidates reviewable evidence. Across CWRU, Ottawa, and SEU, it exposes invalid skeletons, assigns explicit fates to non-full-evaluated candidates, and retains useful final-test performance.
- Conclusion: Candidate-fate accounting addresses the generated-candidate audit gap in automated diagnostic search.It converts otherwise omitted candidate outcomes into auditable evidence.
- Conclusion: Merging repeated trace records, recording legality and allocation rationales, and checking ledger closure with Δclose make candidate outcomes reviewable.The framework covers rejected, skipped, cached, and unfitted candidates.
- Conclusion: Across CWRU, Ottawa, and SEU, the framework exposes invalid skeletons and attributes non-full-evaluated candidates to explicit fates.These mechanisms support complete review of generated candidates beyond fitted trials.
- Conclusion: The framework retains useful final-test performance across the three diagnostic datasets.The passage reports this result for CWRU, Ottawa, and SEU without specifying a numerical value.
Generative AI (GenAI) Usage Disclosure
The authors used generative AI only for limited language polishing and code debugging/checking, not for data generation, result alteration, or scientific decisions. They reviewed and verified the code, results, technical claims, and final manuscript text.
- Usage scope: Generative AI tools supported manuscript language polishing and code debugging/checking in a limited assistive capacity.The tools were not used to generate experimental data, alter results, or make scientific decisions.
- Author verification: The authors reviewed and verified all code, results, technical claims, and final manuscript text.