PaperlayerView exact revision on arXiv

Source-linked AI summary

Protocol effects on feature-based hardware-Trojan detection across Trust-Hub families

Hang Xiao, Chuhong Xu, Kainan Zhou, Gangzhen Qian, Lu Yi

arXiv:2609.07199v1cs.LGcs.AIcs.CEcs.DBcs.GT

TL;DR

Trust-Hub reuses host circuits across Trojan variants, making it possible for training and test folds to share host logic. The paper measures this effect by changing the test boundary while keeping the evaluation pipeline fixed. Random forest and XGBoost lose F1 and AP under family holdout, supporting a narrow conclusion about apparent transfer on this benchmark.

  • Problem

    Trust-Hub contains Trojan-inserted versions of the same host design, so random gate splits can expose a detector to the same circuit on both sides and netlist holdouts may retain sibling variants.

    Source: §I · p. 1

  • Method

    The study compares pooled gate-level cross-validation, netlist holdout, and host-family holdout while keeping the parser, feature matrix, classifiers, weighting rules, and metrics unchanged.

    Source: §I · p. 1

  • Results

    Across 16 Trust-Hub netlists, random forest and XGBoost lose both F1 and AP when a base family is held out, and feature, seed, parser, and reduced-sample checks preserve that direction.

    Source: §IX · p. 6

  • Takeaways & Limitations

    Trust-Hub studies with several variants of one host should report family-aware holdouts, all five family results, pooled scores, and the fold manifest.

    Source: §IX · p. 6

    Only five host families are available, and the resampling and seed runs describe this corpus rather than supporting a population-level claim about arbitrary circuits.

    Source: C. Statistical conclusion validity · p. 6

Abstract

from arXiv · show

Trust-Hub reuses host circuits: several files differ mainly in the inserted Trojan. When gates from sibling variants enter both training and test folds, a detector can benefit from host logic it has already seen. We measure that effect instead of proposing another classifier. The corpus contains 49,124 gates from 16 netlists grouped into five host families. We left the parser, 36 gate features, class weighting, model settings, threshold, and family-level aggregation unchanged and altered one choice: the test boundary. The three settings draw test gates from the pooled corpus, withhold a complete netlist, or withhold every variant of one host. The choice matters. Random forest records F1/AP of 0.914/0.978 with pooled gates, 0.636/0.851 with one netlist held out, and 0.460/0.577 with a host family held out. XGBoost falls from 0.946/0.976 to 0.464/0.544 across the same comparison. Logistic regression loses AP, although its fixed-threshold F1 is not monotonic. Each family shows the same pooled-to-family direction. Feature removal, repeated model and simulator seeds, score normalization, parser-related exclusions, and a smaller sample change the size of the gap without reversing it. Aggregation also matters: a gate-weighted average is dominated by the larger ISCAS files, so the headline values give each host family one vote. Bootstrap and jackknife summaries keep the gap positive, but their folds reuse training families. We treat the five family rows as descriptive evidence rather than independent trials. Five host families are too few for a population claim, and the experiment says nothing about transfer to a new cell library or an industrial design. It supports a narrower conclusion: sibling benchmark variants can inflate apparent transfer. Benchmarks with several variants of one host circuit should report family-aware holdouts and all five family results beside pooled scores.

I. INTRODUCTION

The paper studies whether Trust-Hub’s reused host logic affects reported gate-level Trojan detection. It compares split protocols while keeping the detection pipeline unchanged.

I. INTRODUCTION

  • A dormant Trojan can evade ordinary production tests by activating only under a rare internal condition.The paper frames this as a reason to examine earlier detection from netlists.Source: §I · p. 1

A. Feature-based detection

  • Feature-based netlist detectors classify gates or signals using learned features and decision rules, following earlier systems that used hand-designed structural or activity features.Later studies also explored forests, boosted trees, and graph representations; all depend on which circuits cross the training boundary.Source: §I · p. 1, A. Feature-based detection · p. 1

B. Trust-Hub reuse and the study setting

I. INTRODUCTION

  • The study measures host-circuit reuse by comparing pooled cross-validation, LOCO, and LOFO while holding the parser, features, classifiers, weighting, and metrics fixed.LOCO withholds one netlist; LOFO withholds the complete host family.Source: §I · p. 1

B. Trust-Hub reuse and the study setting

  • The experiments assume flattened gate-level netlists, a cell library, and labeled training designs, and classify at cell-instance level from logic visible in the netlist.The evaluation focuses narrowly on the effect of prior exposure to sibling host logic.Source: B. Trust-Hub reuse and the study setting · pp. 1–2

III. DATASET AND FEATURE PIPELINE

The dataset pipeline parses retained Trust-Hub netlists into host families, documents exceptional handling, and records parser checks and sensitivity exclusions.

A. Netlist processing

  • Trust-Hub filenames identify Trojan variants of host designs, with names such as s38584t200 denoting variant t200 of host s38584.The retained families include RS232 and four ISCAS-89 sequential-circuit identifiers.Source: A. Netlist processing · p. 2
  • Three files receive documented handling for a normalized label or undriven inputs, while s38584 variant t300 is excluded because duplicated flattened net names permit multiple graphs.The parser stops on specified unresolved conditions, none of which remains in the 16 retained files; hashes identify the netlists and pin-direction library.Source: A. Netlist processing · p. 2
  • Alternate repairs can change the excluded s38584 topology, so a sensitivity run removes every design needing a repair or unresolved-source assumption.The surrounding connections resolve most replicated-submodule bus-name cases.Source: A. Netlist processing · p. 2
  • Table I presents the retained Trust-Hub benchmarks by base family.The supplied caption establishes the table’s scope, not its individual entries.Source: Table I · p. 2

B. Features and implementation checks

The pipeline builds 36 gate-level predictors and evaluates them under three increasingly separated test boundaries, with the same feature columns and model settings across protocols.

B. Features and implementation checks

  • The 36 predictors comprise 13 structural graph statistics, 16 binary normalized cell-type indicators, and seven functional summaries in [0,1].Hop counts count each reachable instance once and exclude the target gate; boundary distances and unreachable targets are encoded as 64.Source: B. Features and implementation checks · p. 2, Table II · p. 2
  • Activity features use a zero-delay bit-parallel simulation with 1,024 trajectories, four reset cycles, eight warm-up cycles, and 64 measured cycles.The simulation resamples ordinary non-control inputs, ties scan and test controls low, and applies three relaxation sweeps to 53 gates in combinational cycles.Source: B. Features and implementation checks · p. 2
  • The Boolean implementation and parser interface pass tests covering 66 combinational cell types and 11 flip-flop behaviors, but vendor timing behavior is outside the experiment.The harness includes 850 input vectors, 854 scalar assertions, and Icarus cross-checks.Source: B. Features and implementation checks · p. 2

A. Split protocols

  • Pooled validation splits all 49,124 gates, LOCO tests one netlist while sibling variants may remain in training, and LOFO withholds all variants of one family.The protocols reuse the accepted netlists, feature columns, model settings, weighting rule, threshold, and aggregation code; fold membership changes training size.Source: A. Split protocols · pp. 2–3

B. Models and metrics

The evaluation fixes model and weighting choices while changing fold membership, reports threshold-based metrics alongside score ranking, and summarizes results at the family level.

B. Models and metrics

  • Weighting rules are fixed but use training-fold data: LR uses balanced weights, RF recomputes balanced_subsample weights per tree bootstrap, and XGBoost uses the training-fold negative-to-positive ratio.The scaler is fitted only on each training fold, and no test label enters preprocessing, weighting, or fitting.Source: B. Models and metrics · p. 3
  • Trojan recall, false-positive rate, and Trojan-class F1 use a fixed threshold of 0.5, with average precision reported across thresholds.AP complements the operating-point metrics because class weighting affects score scale.Source: B. Models and metrics · p. 3
  • Metrics are calculated within each base family and macro-averaged so RS232 has the same weight as a larger ISCAS family.In mixed CV and LOCO, a family may collect scores from multiple fitted models.Source: B. Models and metrics · p. 3
  • Paired family differences are summarized with a 200,000-draw bootstrap, an exact two-sided sign test, and leave-one-family-out jackknife ranges.Resampling uses the five family results and does not refit the classifiers.Source: B. Models and metrics · p. 3

V. RESULTS

Family-macro scores for the tree models decline as the test split moves from pooled gates to a held-out host family. LOFO recall varies across families, against a background of extensive shared host logic in sibling variants.

A. Protocol-level performance

  • RF F1/AP falls from 0.914/0.978 (pooled) to 0.460/0.577 (family holdout); XGBoost falls from 0.946/0.976 to 0.464/0.544.The intermediate one-netlist-held-out values are 0.636/0.851 for RF and 0.732/0.817 for XGBoost.Source: Table IV · p. 3, A. Protocol-level performance · p. 3, Figure 1 · p. 4
  • All five family differences are positive: pooled-minus-family gaps are 0.454 RF F1, 0.482 XGBoost F1, 0.401 RF AP, and 0.432 XGBoost AP; sign-test p=0.0625.The authors describe the direction as consistent but the sample as too small for a population claim.Source: A. Protocol-level performance · p. 4
  • In leave-one-family-out jackknife, RF F1 ranges 0.358–0.549 and XGBoost F1 0.416–0.558, both positive; LR is non-monotonic at threshold 0.5, while AP declines from 0.778 to 0.609.Source: A. Protocol-level performance · p. 4

B. Family-level variation

  • LOFO recall is highest for s15850 (0.92–1.00) and lowest for s38584 (0.11–0.20).The smaller RS232 family yields recall of 0.07 for RF, 0.29 for XGBoost, and 0.63 for LR.Source: Figure 2 · p. 4, B. Family-level variation · p. 4
  • Among four multi-variant families, 46,626 of 46,628 benign gates share names with sibling instances, indicating extensive overlap in host logic.The paper says this near-duplicate host logic explains why pooled and circuit-level splits can look stronger than a family holdout.Source: B. Family-level variation · p. 4
  • Family-macro averaging gives every host equal weight; for single-netlist s15850, LOCO and LOFO use the same test file.In other listed multi-variant families, LOCO removes a test file but leaves sibling host logic in training.Source: B. Family-level variation · p. 4

C. Feature and sensitivity checks

Feature restrictions and sensitivity checks change score levels, but the pooled-to-family-held-out ordering persists across the reported feature, seed, normalization, and sample changes.

C. Feature and sensitivity checks

  • Structural features retain the pooled-to-LOFO ordering for both tree models, while seven activity features perform poorly under every split.Removing cell-type indicators changes RF little and lowers XGBoost under LOFO.Source: Table V · p. 4, C. Feature and sensitivity checks · p. 4
  • Removing the feature that identifies all 30 half-adders as Trojan gates changes RF F1 by at most 0.006 and XGBoost F1 by at most 0.016.All 30 half-adders occur in a single s38584 variant, and this feature does not inflate one protocol consistently.Source: C. Feature and sensitivity checks · p. 4
  • Across activity seeds 7, 17, and 29, XGBoost keeps the same protocol order; the largest within-protocol standard deviations are 0.011 F1 and 0.007 AP.Mixed-to-LOFO gaps remain 0.482–0.500 for F1 and 0.420–0.432 for AP.Source: C. Feature and sensitivity checks · p. 4
  • Across model seeds 0–4, all 20 model-metric-seed comparisons retain the same ordering; the largest standard deviations are 0.020 F1 and 0.010 AP.LOFO F1 ranges from 0.443 to 0.469 for RF and from 0.419 to 0.473 for XGBoost.Source: C. Feature and sensitivity checks · p. 4
  • Tie-preserving within-fold percentile normalization leaves the protocol order and mixed-to-LOFO gap intact, while LOCO changes most.Normalized RF AP is 0.978/0.807/0.577 and XGBoost AP is 0.978/0.755/0.544.Source: C. Feature and sensitivity checks · p. 4
  • After three designs with documented repair or unresolved-input assumptions are removed, both tree models retain the mixed/LOCO/LOFO downward order.The reduced set contains 13 designs, 36,892 gates, 211 positives, and all five families; RF F1 is 0.680/0.523/0.358 and XGBoost F1 is 0.892/0.607/0.429.Source: C. Feature and sensitivity checks · p. 5

VI. DISCUSSION

The discussion distinguishes reuse-permitting splits from family holdouts and explains how aggregation and shared structural features shape interpretation. It recommends reporting both split outcomes and the family-level results.

A. Reading pooled and held-out scores

  • Gate-weighted scores are dominated by larger ISCAS designs, whereas family-macro scores give RS232 and s15850 the same influence as s38584.Table IV uses family-macro values, while stored out-of-fold scores permit either aggregation without refitting.Source: A. Reading pooled and held-out scores · p. 5
  • LOCO removes a test netlist but leaves sibling host logic in training for several families; LOFO removes the full family.For s15850, which has one netlist, LOCO and LOFO use the same test file.Source: A. Reading pooled and held-out scores · p. 5
  • Structural features can survive Trojan insertion and be reused in combination by trees; AP falls as Trojan gates rank below benign gates.Structural-only runs retain the gap, while activity-only features do not provide a competitive detector.Source: A. Reading pooled and held-out scores · p. 5
  • Reports should place pooled and family-held-out results side by side and expose all five family rows, alongside threshold, averaging unit, and sibling-split status.A learned representation does not remove the need to define the training boundary.Source: A. Reading pooled and held-out scores · p. 5

B. Operational interpretation

Deployment interpretation depends on the novelty expected after release: a known host family and a new host call for different test boundaries. The interpretation is limited by what the Trust-Hub family labels and gate-level setup represent.

B. Operational interpretation

  • Deployment thresholds must reflect review capacity, false-positive tolerance, and the target synthesis flow, which Trust-Hub does not specify.Lowering a threshold may recover recall on a held-out family but cannot restore AP ranking; deployment therefore needs target-domain calibration data and a split matching expected novelty.Source: B. Operational interpretation · p. 5
  • A pooled split screens another Trojan variant of a known host, whereas a family holdout tests a host whose logic was absent from training.Neither score substitutes for the other; calibration data should match the intended library and synthesis flow.Source: B. Operational interpretation · p. 5
  • GNN4TJ predicts from RTL data-flow graphs and TrojanSAINT samples gate-level subgraphs, so applying a common family boundary would address a shared host-reuse question.Their representations and prediction tasks differ from this gate-level setup, so absolute scores are not directly comparable.Source: B. Operational interpretation · p. 5

A. Construct validity

  • Trojan labels include trigger, payload, and associated inserted logic, while named-host families cannot capture unrecorded shared scripts, library cells, or repeated IP blocks.A narrower label boundary would change prevalence and the meaning of gate-level errors.Source: A. Construct validity · p. 5

B. Internal validity

The reported checks support the implemented parser, features, and within-corpus protocol ordering, but several design and sample boundaries constrain what the results establish. In particular, five related host families do not support population-level claims.

B. Internal validity

  • The parser explicitly handles repeated payload-stage patterns and undriven references, while excluding an ambiguous s38584 variant; removing other affected designs preserves tree-model ordering.The original intent of the ambiguous identifiers remains unknown.Source: B. Internal validity · p. 5
  • The functional features rely on manually specified Boolean semantics, and zero-delay simulation omits hazards and analog timing while approximating combinational cycles.Scalar tests and Icarus cross-checks validate implementation, not vendor-library timing behavior; structural-only runs avoid this uncertainty.Source: B. Internal validity · p. 5

C. Statistical conclusion validity

  • Activity and model seed changes preserve protocol ordering, but five LOFO folds reuse four training families and are not independent experiments.Bootstrap, jackknife, sign-test, and seed results describe this corpus rather than arbitrary circuits.Source: B. Internal validity · p. 5, C. Statistical conclusion validity · p. 6
  • With Trojan gates at 0.69% of the sample, a few false positives can sharply move F1, so AP, TPR, and FPR are also reported.Mixed and LOCO AP combine outputs from several fitted models, and percentile normalization changes LOCO magnitude more than protocol ordering.Source: C. Statistical conclusion validity · p. 6

D. External validity

  • The external evaluation covers 16 variants in five Trust-Hub families, one cell library, and fixed Trojans in flattened netlists.Systems-on-chip, other synthesis flows, layout-only modifications, side-channel detectors, and adaptive attackers are outside the experiment.Source: D. External validity · p. 6
  • IC fingerprinting uses physical signatures absent from this study, while ATTRITION examines adaptive Trojan construction rather than fixed Trust-Hub variants.Recent node-wise graph learning offers another model for a future protocol comparison.Source: D. External validity · p. 6
  • New libraries, synthesis flows, and adaptive attackers can alter cell types, logic cones, and targeted features, none of which appears in the five-family split.Independent circuit families and multiple synthesis flows are needed before treating the rows as estimates for industry-scale designs.Source: D. External validity · p. 6

VIII.REPORTING IMPLICATIONS

The reporting recommendations center on making family membership, fold boundaries, predictions, and aggregation inspectable. This lets readers assess whether the headline reflects a broad pattern or a favorable benchmark case.

VIII.REPORTING IMPLICATIONS

  • An explicit family manifest records each filename, host, Trojan variant, synthesis provenance, library, and parser exception rather than inferring membership from prefixes.The family rule fixes membership before fold construction and permits review without rerunning feature extraction.Source: VIII.REPORTING IMPLICATIONS · p. 6
  • Report pooled, LOCO, and LOFO boundaries alongside five family rows so readers can distinguish variant transfer from transfer to an unseen host.LOCO removes one file while retaining possible sibling logic; LOFO removes the host family.Source: VIII.REPORTING IMPLICATIONS · p. 6
  • Raw out-of-fold prediction records preserve model, protocol, fold, circuit, family, gate, label, and score, allowing threshold or aggregation changes without retraining.Input hashes, fold assignments, and repaired or omitted netlists support reproduction of Table IV.Source: VIII.REPORTING IMPLICATIONS · p. 6
  • Family-level metrics average gates within each family before giving the five families equal weight, while gate-level rows permit aggregation at gate, circuit, or family levels.Fold-level metric averages are not interchangeable because folds represent different units across protocols.Source: VIII.REPORTING IMPLICATIONS · p. 6
  • The difficult s38584 transfer, favorable s15850 result, and small RS232 folds show whether the headline is broad or carried by one benchmark.Publishing the records supports later threshold and calibration studies using existing classifiers and features; raw scores also expose shifts between folds.Source: VIII.REPORTING IMPLICATIONS · p. 6

IX. CONCLUSION

Across 16 Trust-Hub netlists, RF and XGBoost lose both F1 and AP when a base family is held out. The conclusion is limited to sibling benchmark variants, which can make detector transfer look stronger than it is.

IX. CONCLUSION

  • Across 16 Trust-Hub netlists, RF and XGBoost lose both F1 and AP when a base family is held out.Feature, seed, parser, and reduced-sample checks preserve that direction.Source: §IX · p. 6
  • Sibling variants can make detector transfer look stronger than it is, but the result is not a general claim about new hardware.The five family rows are reported as descriptive evidence, not independent trials.Source: §IX · p. 6
  • Future Trust-Hub studies should report pooled and LOFO scores, all five family rows, and the fold manifest.These items expose whether the same host logic appears on both sides of a split and help distinguish benchmark reuse from a representation gain.Source: §IX · p. 6
Loading 2609.07199v1…