PaperlayerView exact revision on arXiv

Source-linked AI summary

FeatureHospital: A Skill-Driven Multi-Agent Framework for Automated Algorithm Customization in Multi-View Multi-Label Feature Selection

Junxuan Li, Zhiqi Chen, Yuzhou Liu, Peng Zhang, Huaxiao Liu

arXiv:2608.16148v1cs.AI

TL;DR

FeatureHospital addresses the recurring expert burden of tailoring multi-view multi-label feature selection algorithms to heterogeneous datasets by using skill-driven agents to diagnose data issues and construct dataset-specific objectives. Across seven datasets, it achieves competitive and consistent performance against seven representative methods.

  • Problem

    Tailoring feature selection algorithms to new datasets remains difficult because it requires dataset understanding and specialized algorithm-design knowledge.

  • Method

    FeatureHospital uses specialized skill-equipped agents to diagnose dataset issues, select problem-specific loss terms, and reconcile them into a compact objective.

  • Results

    FeatureHospital achieves strong, consistent performance across seven datasets, often ranking first or second and outperforming all compared methods on mfeat and 3sources.

  • Takeaways & Limitations

    The framework constructs different feature selection algorithms for different datasets while maintaining competitive effectiveness across dataset changes.

  • Takeaways & Limitations

    Dataset-specific algorithms are limited to selecting and combining components from a fixed, pre-defined Skill library.

Abstract

from arXiv · show

Multi-view multi-label feature selection aims to identify a compact and informative feature subset from heterogeneous views while preserving discriminative information for multiple labels. Existing methods are generally developed from specific modeling perspectives and incorporate mechanisms tailored to particular data characteristics. Designing suitable feature selection algorithms across datasets with diverse and heterogeneous characteristics still relies heavily on expert knowledge and substantial manual effort, imposing considerable time and labor costs that severely hinder the practical adoption of feature selection. To address this problem, we propose FeatureHospital, a Skill-driven multi-agent framework for automated multi-view multi-label feature selection algorithm design. FeatureHospital first diagnoses the target dataset to identify its feature selection issues. Based on the diagnosis, specialist agents equipped with domain Skills then prescribe corresponding optimization strategies and Loss terms for different issues. After that, the resulting prescriptions are reconciled to remove overlaps and resolve conflicts before being integrated into a compact dataset-specific objective. Finally, the constructed objective is optimized to select the final feature subset. Experimental results demonstrate that FeatureHospital can construct effective feature selection algorithms for different datasets based on their individual characteristics.

Introduction

The introduction frames multi-view multi-label feature selection as a heterogeneous, challenge-rich problem whose practical algorithm design remains labor-intensive. It presents FeatureHospital as a Skill-driven multi-agent framework that diagnoses datasets and automatically constructs tailored feature-selection objectives.

  • Motivation: Multi-view multi-label data combine heterogeneous feature views with multiple semantic labels and may exhibit view-quality imbalance, feature redundancy, missing labels, and label imbalance.These characteristics create complex challenges for feature selection in real-world scientific applications.
  • Motivation: Researchers face a recurring algorithm-design burden because selecting or designing a suitable feature-selection method requires understanding dataset characteristics and specialized knowledge.The introduction presents feature selection as a practical tool that often demands substantial manual analysis and redesign.
  • Motivation: Reusable procedural Skills enable general-purpose agents to act as domain experts, motivating automated dataset-tailored feature-selection algorithm design.The framework builds on recent advances in LLM-based agents and Skills that encode domain knowledge and structured tool-use procedures.
  • FeatureHospital: FeatureHospital automates the process from dataset diagnosis to algorithm construction, generating feature-selection strategies specific to each dataset.Its workflow diagnoses feature-selection issues, uses specialized Skill-equipped agents to prescribe strategies, constructs an optimization objective, and performs training and feature selection.

Related Work

Related work shows that MVML feature selection must handle heterogeneous views and multiple labels through dataset-specific modeling, while Skill-driven multi-agent systems offer specialization for complex tasks. FeatureHospital bridges these directions by using specialized Skills and agents to diagnose dataset issues and collaboratively design tailored feature selection algorithms.

  • MVML Feature Selection: MVML learning combines heterogeneous views and multiple labels to capture complementary semantic information, but this richness increases data complexity.Feature selection therefore seeks a compact, discriminative subset.
  • MVML Feature Selection: Existing MVML feature selection methods address different data structures through specific modeling perspectives, including view-specific or hybrid label information.Other approaches construct cross-view representations through global-view reconstruction or embedded feature fusion.
  • Skill-Driven Multi-Agent Systems: LLM-based multi-agent systems coordinate agents with complementary roles, while Skill-based approaches provide reusable, composable domain specialization without retraining.General-purpose agents may still lack specialized knowledge and procedural experience for different domains.
  • FeatureHospital: FeatureHospital analyzes target datasets, assigns identified problems to specialist agents, selects corresponding losses, reconciles conflicts and redundancy, and optimizes a dataset-specific objective.These stages form the framework’s prescribed workflow for constructing feature selection algorithms.
  • FeatureHospital: Skill-driven multi-agent systems remain largely unexplored for feature selection, motivating FeatureHospital for MVML feature selection.The framework equips specialized agents with feature selection Skills to diagnose dataset-specific issues, translate them into optimization strategies, and collaboratively design algorithms.

Methodology

FeatureHospital builds dataset-specific feature-selection objectives through a staged multi-agent workflow. It analyzes the training data, diagnoses issues, routes them to specialist Departments, reconciles prescribed losses, and optimizes the resulting objective to select features for test-set evaluation.

  • Dataset setup: FeatureHospital partitions each dataset into disjoint training and test subsets, using only Dtr during algorithm construction before final evaluation on Dte.The selected feature subset is evaluated on the test set after optimization.
  • Dataset diagnosis: Executable analysis tools construct a Statistical Profile covering data scale, feature budget, label distribution, feature quality, redundancy, label dependency, and view heterogeneity.The Diagnosis Agent interprets this profile with Diagnostic Skills and produces Issue Cards containing issues, supporting abnormal metrics, and diagnostic summaries.
  • Issue triage: The Triage Doctor compares Issue Cards with Department scopes, activates relevant Departments, and routes each identified issue to appropriate specialists.This routing is guided by the Triage Skill and produces a complete assignment of issues to activated Departments.
  • Specialist consultation: Specialist Doctors select applicable Loss Medicines for assigned issues, with each medicine specifying an implemented loss term, intended effect, and applicability conditions.Selections from all activated Departments are collected for subsequent objective construction.
  • Objective construction: The Pharmacist reconciles prescribed medicines by removing redundancy or conflicts, assigning functional roles, and controlling their contributions in a unified objective.Roles include backbone, supporting, regularizer, and guardrail losses; the final objective contains 1 + Nr + Ns + Ng loss terms.
  • Feature optimization: FeatureHospital optimizes a continuous feature-selection vector on the constructed objective, ranks features by selection strength, and returns the top-k subset.The trainable parameters are instantiated as a logit vector transformed by a sigmoid function with temperature parameter τ.

Experiments

Experiments on seven datasets across five domains evaluate FeatureHospital against representative feature-selection baselines using four multi-label metrics. Results show strong cross-dataset performance, stable selected weights, and benefits from complementary losses, LLM decisions, and staged multi-agent collaboration.

  • Experimental setup: Experiments use seven benchmark datasets spanning handwritten digits, images, news, biology, and music, with 70%/30% train-test splits repeated ten times.Results are averaged over feature-selection ratios from 2% to 20% at 2% intervals and reported as mean ± standard deviation.
  • Experimental setup: FeatureHospital is compared with seven methods, including DHLI, EF2FS, ENM, GRAFS, I2VSLC, MSFS, and the LLM-based LLM-Select.Evaluation uses Average Precision, Area Under the ROC Curve, Ranking Loss, and Zero-one Loss.
  • Overall comparison: FeatureHospital achieves the best or second-best results on multiple datasets and consistently outperforms all compared methods across AP, AUC, RL, and ZL on mfeat and 3sources.The reported results indicate strong performance and consistent effectiveness across dataset changes.
  • Parameter sensitivity: The weights selected for the yeast objective lie within stable regions, although Negative Dependency Separation is more sensitive to excessively large values.Sensitivity analysis varies each term weight over {0.001, 0.01, 0.1, 1, 10, 100, 1000} while fixing the others.
  • Ablation study: Removing non-backbone terms degrades performance, confirming that the additional loss terms provide effective complementary constraints.The ablation compares the Backbone Only variant with the full objective.
  • Ablation study: The full method generally outperforms random consultation, pharmacist removal, rule-based decisions, and a single-agent pipeline, supporting staged role specialization and LLM involvement.Rule-based replacement performs particularly worse on mfeat and 3Sources, while collapsing the process into one LLM call yields consistently lower performance.

Conclusion … B Additional Experiments

FeatureHospital automates dataset-specific multi-view multi-label feature-selection algorithm customization by diagnosing dataset issues, selecting problem-specific Loss Medicines, and reconciling them into compact objectives. The paper reports competitive and consistent results across seven datasets, while identifying reliance on human-curated Skills and the absence of experience-driven self-evolution as limitations.

  • Conclusion: FeatureHospital diagnoses dataset issues, selects problem-specific Loss Medicines, reconciles them into compact objectives, and constructs feature-selection algorithms for different datasets.Experiments on seven datasets demonstrate competitive and consistent performance, and ablation studies validate objective components.
  • A.1 Threats: FeatureHospital currently depends on predefined Skills constructed by human experts, including applicability conditions, procedures, and candidate Loss Medicines.This provides a controllable and interpretable foundation for automated algorithm customization.
  • A.1 Threats: FeatureHospital does not retain diagnostic decisions, Department routing, selected Loss Medicines, objectives, or empirical outcomes to improve subsequent design processes.Consequently, it does not yet accumulate experience across repeated runs.
  • A.2 Discussion: Automatic Feature Selection: AutoML seeks to automate machine-learning solution construction, reducing expert effort in algorithm design, selection, and configuration.Applied to feature selection, this becomes automatic construction of algorithms tailored to target-dataset characteristics.
  • A.2 Discussion: Automatic Feature Selection: Existing automatic feature-selection studies primarily use algorithm recommendation and pipeline search based on dataset meta-features and historical performance.Meta-learning frameworks recommend suitable algorithms or configurations for new datasets.
  • A.2 Discussion: Automatic Feature Selection: FeatureHospital differs by decomposing target-dataset requirements into issues such as label imbalance, feature redundancy, and view-quality imbalance.It then selects a corresponding optimization component for each identified issue rather than directly choosing a complete predefined algorithm or pipeline.

B.1 Backbone LLM Replacement

FeatureHospital remains effective when its GPT-5.5 backbone is replaced with models spanning capability levels and accessibility settings, showing limited sensitivity to backbone choice. Smaller backbones remain competitive, while GPT-5.5 offers the most stable and balanced averages; one pipeline execution costs approximately $0.08–$1.31 and is performed only once per dataset.

  • Backbone robustness: The evaluated backbones include proprietary high-capability, open-source flagship, proprietary compact, and other accessibility settings, replacing the GPT-5.5 reference backbone.The passage names Claude Sonnet 5, Gemini 3.1 Pro Preview, DeepSeek V4-Pro, and GPT-5.4 mini among the alternatives.
  • Backbone robustness: Across six backbones, best-to-worst average differences are 0.0091 for AP, 0.0078 for AUC, 0.0066 for RL, and 0.0143 for ZL.No replacement caused substantial performance degradation or evaluation failure across seven datasets.
  • Backbone robustness: GPT-5.4-mini achieves average AP, AUC, RL, and ZL values of 0.6903, 0.6874, 0.2344, and 0.8177, respectively, remaining close to GPT-5.5.Qwen3.5-27B also achieves competitive averages and performs best on SCENE under several metrics.
  • Backbone robustness: GPT-5.5 achieves the best average AP, AUC, and RL values of 0.6938, 0.6928, and 0.2308, respectively, plus the second-best average ZL of 0.8143.Other backbones occasionally lead on individual datasets, but none consistently dominates across datasets and metrics.
  • Execution cost: A complete pipeline execution costs approximately $0.08 to $1.31 on average and constructs a dataset-specific objective and algorithm configuration without repeated LLM-based design.The cost estimate uses regular real-time API prices without prompt caching, batch processing, or promotional discounts.

B.2 Statistical Significance Analysis

FeatureHospital achieves the best average rank across all four evaluation metrics, with Friedman tests indicating significant differences among methods. Nemenyi tests confirm significant advantages over several methods, while limited datasets reduce the power of some pairwise comparisons.

  • Statistical significance: Friedman tests reject equivalent-performance null hypotheses for AP (p = 0.0012), AUC (p = 0.0014), RL (p = 0.0010), and ZL (p = 0.0407).The analysis follows Friedman testing followed by Nemenyi post-hoc pairwise comparisons.
  • Average ranking: FeatureHospital achieves the best average rank for AP, AUC, RL, and ZL, with ranks 2.000, 1.571, 2.000, and 1.857, respectively.The rankings are computed across eight compared methods and seven datasets.
  • Pairwise comparisons: FeatureHospital significantly outperforms EF2FS and MSFS on AP and RL, DHLI, EF2FS, and MSFS on AUC, and MSFS on ZL.These conclusions come from the Nemenyi post-hoc pairwise comparisons.
  • Pairwise comparisons: Differences between FeatureHospital and LLM-Select, I2VSLC, ENM, and GRAFS do not exceed the critical difference on any metric.Therefore, these pairwise differences are not statistically significant under the Nemenyi test.
  • Limitations: The seven-dataset analysis yields a large critical difference, limiting the statistical power of some pairwise comparisons, while FeatureHospital remains competitive without substantial metric degradation.The critical difference is calculated for eight methods and seven datasets at α = 0.05.

B.3 Performance across Feature Selection Ratios

Across feature selection ratios from 2% to 20%, FeatureHospital generally maintains leading or competitive performance and avoids consistent degradation as the feature budget changes. Its performance is strong at small ratios, often stabilizes after moderate selection, and varies smoothly across consecutive ratios.

  • Evaluation setting: Across ratios from 2% to 20%, FeatureHospital is evaluated using AP, AUC, RL, and ZL on six datasets beyond yeast.The main text reports yeast curves separately as a representative example.
  • Overall performance: FeatureHospital generally maintains leading or competitive performance across ratios, with especially clear advantages on mfeat and 3Sources.It remains close to the best-performing methods on other datasets, despite isolated wins by individual baselines.
  • Performance versus selection ratio: Performance generally improves or remains stable as more features are selected, while strong results at small ratios indicate effective compact subsets.After a moderate number of features is selected, performance often approaches a stable region because the most useful information has already been retained.
  • Robustness across feature budgets: FeatureHospital exhibits relatively smooth curves across consecutive ratios, whereas several competing methods show substantial fluctuations when the selected-feature count changes slightly.This indicates lower sensitivity to the exact feature budget and more consistent subsets across ratios.

B.4 Complete Ablation Results · C Skills & Prompt

Complete ablations show that FeatureHospital’s gains arise from combining dataset-specific objective components, structured multi-agent decisions, LLM-based adaptation, and objective reconciliation. Its prompts and Skills constrain agents to diagnose confirmed issues, select implemented capabilities, and produce validated prescriptions for dataset-specific objective construction.

  • B.4 Complete Ablation Results: FeatureHospital achieves the best average performance across all four metrics, with 0.6938 AP, 0.6928 AUC, 0.2308 RL, and 0.8143 ZL.The complete method remains strongest on average across the seven datasets.
  • B.4 Complete Ablation Results: The Backbone Only variant drops to 0.6847 AP and 0.6861 AUC, while RL and ZL worsen to 0.2363 and 0.8275.Its limitations are especially evident on mfeat and 3Sources, where dataset-specific Loss Medicines add constraints beyond feature–label relevance.
  • B.4 Complete Ablation Results: Random Triage and Random Consultation both perform worse than the complete method under all four metrics, with larger standard deviations in some mfeat and MIRFlickr results.Random Triage can activate unrelated Departments, while Random Consultation can select Loss Medicines mismatched to assigned problems.
  • B.4 Complete Ablation Results: All Rule-Based Decisions achieves 0.6781 AP, 0.6843 AUC, 0.2429 RL, and 0.8354 ZL, producing the weakest or nearly weakest average performance on most metrics.The results indicate that fixed rules cannot fully capture context-dependent interactions among dataset characteristics, diagnostic evidence, and candidate Loss Medicines.
  • B.4 Complete Ablation Results: Single LLM Agent performs consistently worse on average, while Without Pharmacist is the strongest ablation but remains inferior to the complete method on every metric.The findings support specialized agents, staged collaboration, and pharmacist reconciliation to remove overlaps, resolve conflicts, and balance independently prescribed objectives.
  • B.4 Complete Ablation Results: The framework’s effectiveness arises from the combined contributions of dataset-specific components, structured triage and consultation, LLM-based decisions, multi-agent specialization, and final objective reconciliation.Critical-difference diagrams compare AP, AUC, RL, and ZL ranks across seven datasets using the Nemenyi test at α = 0.05 with CD = 3.968.
  • C Skills & Prompt: Runtime requests combine stage instructions, static Skill knowledge, dataset-specific context, and output contracts, while prompts enforce JSON-only outputs and implemented capability boundaries.Department Doctors receive domain knowledge, dataset summaries, activated cases, capability catalogs, parameter ranges, schemas, and prior validation errors; the Pharmacist merges prescriptions into one conflict-aware objective without re-diagnosing.
  • C Skills & Prompt: A prompt defines the role, task, constraints, and output schema, whereas a Skill specifies applicability, procedure, tool interactions, stopping conditions, and validation rules.All six specialist Skills use dataset-independent loss catalogs, and LLMs may select only implemented entries rather than synthesize new losses.

C.1 Dataset Analysis

Dataset analysis deterministically computes statistics from (Xtr, Ytr) and uses the LLM only to phrase rule-confirmed findings, not to determine abnormalities.

  • C.1 Dataset Analysis: Dataset analysis computes deterministic statistics from (Xtr, Ytr), then passes rule-confirmed findings to the LLM for concise diagnostic wording.The LLM does not decide whether an abnormality exists at this stage.

D Implementation Details · E Case Study

FeatureHospital optimizes a continuous feature mask with fixed settings, ranks features under a 20% budget, and evaluates selected subsets across repeated train-test splits. A traceable SCENE case study follows dataset analysis, triage, specialist consultation, and objective construction as one continuous chain.

  • D Implementation Details: The continuous mask z is parameterized as sigmoid(a/T) with T = 1.0 and optimized using Adam for 300 iterations.Logits are initialized from relevance-based feature scores, with learning rate 0.03, β1 = 0.9, β2 = 0.999, and ϵ = 10−8.
  • D Implementation Details: No additional hyperparameter tuning is performed after objective construction.
  • D Implementation Details: For each random split, the objective is optimized once with a maximum feature budget of 20% of the original features.Features are ranked by learned mask values, and subsets at ratios from 2% through 20% are obtained as ranking prefixes using max(1, ⌊rd⌋) selected features.
  • D Implementation Details: Each dataset is randomly divided into 70% training and 30% test data using ten fixed random seeds from 0 to 9.Prediction uses a multi-output 10-nearest-neighbor classifier, with samples lacking positive labels and labels lacking positive training examples excluded per split.
  • D Implementation Details: Performance is reported with Average Precision (AP), macro-averaged AUC, Ranking Loss (RL), and exact-match zero-one loss (ZL).Metrics are first averaged over ten feature-selection ratios, then summarized by mean and sample standard deviation over ten random splits; higher AP and AUC are preferred, while lower RL and ZL are preferred.
  • E Case Study: The SCENE case study traces dataset analysis, triage, specialist consultation, and objective construction as one continuous chain.SCENE contains several independently confirmed conditions, and numeric values are rounded while issue names, decisions, and weights are copied from persisted main-experiment artifacts.

E.1 Dataset Analysis Trace · E.2 Department Triage Trace

The traces show that FeatureHospital accurately diagnoses severe label-distribution problems from outer-training statistics and routes related findings to a coherent department without duplicating treatments. Triage preserves independent targets, absorbs consequential evidence, and retains beneficial or hierarchical context appropriately.

  • E.1 Dataset Analysis Trace: Only the 3,080 outer-training samples were profiled, excluding all 1,320 held-out samples from the reported statistics.
  • E.1 Dataset Analysis Trace: SCENE contains 634 features, 33 labels, a minimum frequency of 0.000649, and a maximum frequency of 0.968182.The extrema correspond to 2 and 2,982 positive samples, respectively.
  • E.1 Dataset Analysis Trace: Independent statistics support the diagnosis through frequency extremes, a 1,491.0 imbalance ratio, a 0.6526 frequency Gini, and 13/33 rare labels.The evidence explicitly includes a label with only two positives and another present in more than 96% of samples.
  • E.1 Dataset Analysis Trace: The diagnosis identifies EXTREME_LABEL_FREQUENCY, LABEL_IMBALANCE, and RARE_LABELS as severe findings.It summarizes extreme rare/common labels, a very large imbalance ratio, many rare labels, and a minimum positive count of two.
  • E.2 Department Triage Trace: LABEL_IMBALANCE is activated with problem strength 0.90 and confidence 1.00, while related findings are routed to the same owner.The routed issues include EXTREME_LABEL_FREQUENCY, RARE_LABELS, LOW_SAMPLE_PER_LABEL, FEATURE_BUDGET_PRESSURE, and weak minor-label signal.
  • E.2 Department Triage Trace: FEATURE_QUALITY_DEFECT is absorbed by VIEW_QUALITY_IMBALANCE because sparsity and quality defects are view-specific, whereas LOCAL_LABEL_INCONSISTENCY remains context-only.The profile instead confirms beneficial LOCAL_LABEL_CONSISTENCY.
  • E.2 Department Triage Trace: Feature-budget pressure and weak minor-label signals are absorbed into LABEL_IMBALANCE to avoid duplicated objectives while preserving downstream evidence.

E.3 Specialist Diagnosis Trace · E.4 Pharmacist Fusion Trace

The SCENE traces show specialists diagnosing view-specific relevance comparability before prescribing calibration, while the Pharmacist fuses overlapping prescriptions into a compact objective using training-only mechanical probes. The final objective retains distinct relevance, redundancy, separation, and budget roles while excluding redundant or weakly supported terms.

  • E.3 Specialist Diagnosis Trace: SCENE’s diagnosis asks which treatment matches the observed form of view heterogeneity, rather than merely recognizing that the dataset is multi-view.
  • E.3 Specialist Diagnosis Trace: 0.4163 zero-rate gap and 1.1055 view scale-heterogeneity gap indicate substantial cross-view differences in sparsity and scale.The views contain 64, 225, 144, 73, and 128 features, with zero rates [0.00026,0.00231,0.41651,0.01198, 0.00029].
  • E.3 Specialist Diagnosis Trace: The Doctor selected view_normalized_relevance_loss with weight 0.30, moderate quality weighting, and score clipping because raw relevance scores were not cross-view comparable.adaptive_view_allocation_loss and view_coverage_floor_loss were not selected; normalization addressed comparability without forcing uniform allocation or a hard view floor.
  • E.3 Specialist Diagnosis Trace: The calibration prescription used positive and negative evidence from the same 3,080-sample outer-training partition, rejecting mechanically broader catalog activation.The selected loss targeted score comparability, while allocation and floor terms lacked evidence that their constraints were needed.
  • E.4 Pharmacist Fusion Trace: The Pharmacist converted overlapping local prescriptions into a compact global objective using Doctors’ rationales and low-cost training-only probes, without classifier metrics.
  • E.4 Pharmacist Fusion Trace: The fused objective retains label_weighted_relevance_loss at 1.20, redundant_cluster_quota_loss at 0.12, view_normalized_relevance_loss at 0.30, and separation and budget guardrails at 0.05 and 0.2625.The retained terms are respectively primary, regularizer, supporting, guardrail, and guardrail components.
  • E.4 Pharmacist Fusion Trace: 0.5874 endpoint damage and 28.57% top-k change supported retaining cluster redundancy, whereas label-contribution balance changed none of the top-k set and only 2.19 × 10−5 endpoint.Graph-smoothed and view-normalized relevance had gradient cosine 0.9699; raising budget weight from 0.15 to 0.2625 improved its endpoint by 0.0237 with 98.43% top-k overlap and 0.0052 primary-endpoint cost.
  • E.4 Pharmacist Fusion Trace: The probes used 2,156 outer-training samples without held-out labels or classifier metrics, supporting fusion reasonableness but not predictive claims.Graph smoothing was merged into the primary relevance backbone plus view calibration, and label-contribution balance was dropped for weak distinct mechanical effect.
Loading 2608.16148v1…