PaperlayerView exact revision on arXiv

Source-linked AI summary

$\mathrm{TRIZ}^{a}$: Guiding Agent Evolution from Pattern Recognition to Solution Invention

Wenyin Liu, Yiheng Huang, Kai Wang

arXiv:2610.04555v1cs.AI

TL;DR

Evaluation-after-the-fact can rank candidates without identifying the contradiction to address or showing whether a gain is genuine. TRIZa combines typed TRIZ transformations with FI/Ideality and hard gates under a frozen contract. Its cybersecurity studies report paired-rerun gains, while a CICIoT2023 candidate improved macro-F1 but was not promoted.

  • Problem

    Evaluation scores can rank candidates but do not identify blocking contradictions, justify transformations, or distinguish technical advances from evaluator exploitation.

    Source: §1 · p. 1

  • Method

    TRIZa pairs typed TRIZ code transformations with FI/Ideality under a frozen reference contract and uses hard gates to govern promotion.

    Source: §1 · pp. 1–2

  • Results

    Cybersecurity studies report paired-rerun gains, while CICIoT2023 macro-F1 rose from 0.8325 to 0.8483 but the candidate was not promoted.

    Source: §4.3 · p. 6, §5 · p. 8

  • Takeaways & Limitations

    The workflow preserves an audit trail from contradiction and transformation through evaluation and selection, including outcomes that are not promoted.

    Source: §5 · p. 8

    The study does not include matched-budget ablations, so it cannot determine how much TRIZ, FI/Ideality, or the controller contributed to observed gains.

    Source: §4.4 · p. 7, §4.5 · p. 8

Abstract

from arXiv · show

We propose $\mathrm{TRIZ}^{a}$ (TRIZ exponentiated by an agent), a general R\&D automation paradigm that combines TRIZ inventive theory with LLM-driven agent evolutionary search. TRIZ's 40 inventive principles and contradiction matrix provide structured, explainable directions for solution generation, replacing random or untyped mutation with theory-guided ideation. Functional information (FI), operationalized under a frozen reference contract, is combined with TRIZ Ideality to measure useful and harmful function on a commensurable information scale, while hard gates keep promotion distinct from metric improvement. We validate $\mathrm{TRIZ}^{a}$ in cybersecurity--an adversarial and rapidly evolving domain--on PowerDuck GOOSE, CICIoT2023, and CIC-DDoS2019. Under paired-rerun protocols with protocol fingerprinting and hard-gate validation, the legacy experiments yield absolute F1 improvements of $+2.88$, $+4.23$, and $+0.15$ percentage points, respectively. A completed 45-activity CICIoT2023 campaign further increases macro-F1 from $0.8325$ to $0.8483$, but does not pass its frozen promotion gate. Every result remains traceable from contradiction identification and TRIZ principle selection to code transformation, evaluation metrics, and promotion decision.

1 INTRODUCTION

Evaluation can rank candidates but does not explain which contradiction to address or whether a gain is genuine; cybersecurity search must also balance detection against adverse effects. TRIZa combines typed TRIZ transformations with FI/Ideality and hard promotion gates under a frozen evaluation contract.

1 INTRODUCTION

  • Evaluation scores alone do not identify blocking contradictions, justify transformations, or distinguish technical advances from candidates exploiting the evaluator.Cybersecurity search must balance detection quality against false positives, missed attacks, leakage, runtime cost, and split contamination under a fixed protocol.Source: §1 · p. 1
  • TRIZ principles become typed code-transformation operators that candidates declare alongside the contradiction before results are known.Operators include Segmentation, Local Quality, Prior Action, and Parameter Change.Source: §1 · pp. 1–2
  • FI/Ideality describes useful and harmful achievement on a common information scale, with the reference population, criteria, and evaluation procedure fixed before search.FI measures functional achievement rather than serving as an independent objective to maximize.Source: §1 · p. 2
  • Hard gates separate metric improvement from promotion, while audit records preserve the contradiction, transformation, protocol, metrics, and decision.Leakage, unsafe operation, or incomplete replication cannot be offset by a higher aggregate score.Source: §1 · p. 2

2 RELATED WORK

TRIZa places TRIZ principles inside an executable code-evolution loop, while related LLM systems use TRIZ mainly to structure ideation or automate broader research activities. Its FI account is tied to a predefined reference population and qualification threshold.

TRIZ and computational invention

  • TRIZa makes a selected TRIZ principle the declared type of a code mutation inside an executable evolution loop, recorded before evaluation.Earlier TRIZ adaptations structure ideation, while recent LLM systems automate TRIZ reasoning or coordinate TRIZ-specialized agents.Source: §2 · p. 2

LLM-guided program evolution

  • Unlike broad research-automation systems, TRIZa constrains each candidate transformation by the identified contradiction, selected TRIZ type, frozen protocol, and evidence obligations.This contrasts with systems that automate research lifecycle activities or couple reproduction, repair, ideation, evaluation, and supervised improvement.Source: §2 · p. 2

Function-conditioned selection

  • FI is the negative logarithm of the reference-population fraction meeting a specified function threshold; rarity alone does not establish usefulness.The paper fixes qualification criteria, reference population, sample size, and stopping rule before evaluation; confidence sequences may support future data-dependent stopping.Source: §2 · pp. 2–3

3 METHODOLOGY

The framework freezes the problem and evaluation contract before theory-guided candidate generation, then evaluates and governs promotion through a closed loop. TRIZ types proposed transformations, functional information compares useful and harmful performance against a fixed reference population, and traceable evidence supports review of decisions.

3.1 FRAMEWORK OVERVIEW

  • A human-in-the-loop brief defines the target and constraints, then a frozen evaluation protocol governs principle selection, candidate generation, and evaluation or review.The four stages are problem formulation, principle selection, solution generation, and evaluation and evolution; high-risk actions require explicit human authorization.Source: §3.1 · p. 3

3.2 THEORY-GUIDED IDEATION

  • Each candidate must declare a contradiction and instantiate a selected TRIZ principle through a typed operator before evaluation; favorable results alone do not establish TRIZ guidance.The operator specifies input and output types, constraints on what may change, and required evaluation evidence; TRIZ directs search but does not guarantee success.Source: §3.2 · pp. 3–4

3.3 FUNCTIONAL INFORMATION AND FI/IDEALITY

  • Useful and harmful function are compared separately against a fixed reference population under evidence-level requirements, with both expressed as information measures.Useful-function information reflects how rarely reference configurations meet or exceed the candidate's qualified performance; harmful-function information separately measures adverse effects.Source: §3.3 · pp. 4–5
  • FI-based Ideality supports diagnosis and tie-breaking only among candidates with the same certification and replication status; it cannot override mandatory gates.The controller checks integrity, safety, reproducibility, and resources before using FI, cost, or Ideality to compare candidates.Source: §3.3 · p. 5

3.4 TRACEABILITY AND EVIDENCE STATE

  • The controller advances an evidence-bearing artifact through four validation levels, and a source diff without protocol and evidence references cannot be promoted.Rejected candidates remain archived with their rejection reason; execution records link artifacts, contradictions, operators, protocol fingerprints, metrics, gate outcomes, and decisions.Source: §3.4 · p. 5
  • Traceability makes proposals and execution inspectable, but does not establish that TRIZ principles caused the observed gain without an equal-budget unguided control.The PowerDuck record links a contradiction and two principles to an implementation and weighted-F1 change, while the authors state this does not prove component-level causality.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments test TRIZa across three cybersecurity tasks using paired protocol checks and distinct evidence layers. Legacy candidates outscored rerun baselines, while formal CICIoT2023 V2 improved several metrics but failed frozen promotion thresholds.

4.1 EXPERIMENTAL SETUP AND PAIRED PROTOCOL

  • The three tasks use distinct registered scopes and metrics, while paired comparisons require matching protocol fingerprints covering data, splits, preprocessing, metrics, seeds, and candidate budget.The harness also checks tests, representation budget, train-test overlap, and campaign-specific safety gates before promotion.Source: §4.1 · p. 6

4.2 LEGACY PAIRED OUTCOMES

  • Under same-protocol reruns, TRIZa candidates scored higher than baselines on all three tasks, with CIC-DDoS covering its complete registered ten-fold scope.The evidence ladder distinguishes these paired legacy outcomes from the later CICIoT value and formal V2 promotion decision.Source: §4.2 · p. 6

4.3 COMPLETED CICIOT2023 V2 AND FI/IDEALITY PROMOTION AUDIT

  • 0.8483 macro-F1 from 0.8325 (+0.0158) accompanied higher attack recall and lower detector FPR, but the frozen V2 candidate missed two promotion thresholds.The 45 registered activities were completed on a frozen candidate; the controller therefore withheld promotion.Source: §4.3 · pp. 6–7
  • 0.811 promotion F1, 0.686 recall, and 0.989 specificity for balanced V2 contrasted with safety-first V2's 0.409 promotion F1, 0.257 recall, and 0.998 specificity.Campaign08 is a descriptive aggregate of 2,040 nested decisions, not independent replicates.Source: §4.3 · p. 7, Figure 2 · p. 7

4.4 DOWNSTREAM EVOLUTION AND NEGATIVE EVIDENCE

  • V17 reached macro-F1 0.8753 and MCC 0.7755, but attack recall was 2.43 percentage points below V1/RF100; V18 was rejected after other metrics deteriorated.V18's recall recovered slightly to 0.8811; the sequence is downstream evolution, not part of formal V2 selection.Source: §4.4 · p. 7

4.5 WHAT THE EVIDENCE ESTABLISHES

  • The evidence supports cross-task application and traceable candidate decisions, but does not isolate the contributions of TRIZ, FI/Ideality, or the controller.Causal attribution would require matched-budget ablations.Source: §4.4 · p. 7, §4.5 · p. 8

4.6 TRACEABILITY AND REPRODUCIBILITY

  • For each candidate, records link the diagnosed contradiction and TRIZ principle to transformation, evaluation, results, gate outcomes, and controller decision.These records allow reported decisions to be checked against measured results.Source: §4.6 · p. 8

5 CONCLUSION

TRIZa combines typed TRIZ search directions, FI/Ideality measures, and hard promotion gates in an auditable agent loop. Its records connect each candidate’s guidance and transformation to evaluation and decision evidence, while manuscript claims remain limited to aggregate evidence.

5 CONCLUSION

  • Hard gates distinguish metric improvement from promotion, and the workflow retains unpromoted or condition-specific results as complete audit records.The conclusion reports a CICIoT2023 candidate that improved three detector measures but was not promoted.Source: §5 · p. 8
  • TRIZa links contradiction identification, TRIZ principle selection, transformation, protocol, measured results, gate outcomes, and controller decisions in an auditable workflow.The audit trail makes each candidate’s guidance, generation, evaluation, and selection reviewable.Source: §5 · p. 8
  • Human authors originated the goal and research directions, while AI coding agents implemented the system, ran experiments, produced figures, and drafted the manuscript.Human authors reviewed and extensively revised the outputs and take responsibility for the final content.Source: §5 · p. 8

A.1 TRACEABILITY RECORD SCHEMA

  • Claims are limited to the manuscript’s aggregate evidence; the planned public artifact package will include traceability files and a SHA256SUMS.txt digest file.The files are not included in the current submission package.Source: §A.1 · p. 10
  • Table 3 identifies the traceability schema’s planned archive-relative filenames.Source: Table 3 · p. 10

A.2 CANONICAL PAPER-LEVEL RECORD

The paper presents a PowerDuck paired observation with a traceable contradiction-to-transformation record, while specifying that the record does not establish principle-level causality. The accompanying mapping defines transformation classes, and the frozen contract makes promotion contingent on registered gates.

A.2 CANONICAL PAPER-LEVEL RECORD

  • The paper-level PowerDuck record is not a raw execution record and omits run identifiers, raw predictions, and file hashes.Source: §A.2 · p. 10
  • The PowerDuck record connects the coverage-versus-few-shot contradiction to Local Quality (3) and Parameter Change (35), without claiming either principle caused the gain.Source: §A.2 · p. 10
  • Weighted-F1 rose from 0.9694 to 0.9982 in the PowerDuck paired rerun, retained as a complete observation with protocol match and integrity gates true.The paper states that this record does not support a principle-level causal claim.Source: §A.2 · p. 10

B FULL TRIZ-TO-TRANSFORMATION MAPPING

  • The TRIZ mapping defines transformation classes, not guaranteed improvements, and concrete operators must specify types, admissible arguments, prohibited changes, and evidence obligations.Source: B FULL TRIZ-TO-TRANSFORMATION MAPPING · p. 10
  • The listed principle classes include Prior Action for precomputation or validation and Parameter Change for changing a type, range, threshold, loss, weight, or operating point.Source: B FULL TRIZ-TO-TRANSFORMATION MAPPING · p. 11

C.1 FROZEN CONTRACT

  • The frozen contract binds dataset, split, preprocessing, budget, metrics, harm events, resource ceilings, stopping rule, and code and environment digests; changing a bound field creates a new protocol.Promotion is inadmissible when any registered gate fails, regardless of FI or Ideality.Source: §C.1 · p. 11

C.2 DATASET AND METRIC CONTRACTS

The formal CICIoT campaign uses registered promotion thresholds that are specific to that campaign, rather than universal constants.

C.2 DATASET AND METRIC CONTRACTS

  • The formal CICIoT strict gate requires at least a 3.2277% relative macro-F1 gain and a 0.005 detector-FPR reduction, in addition to integrity gates.These thresholds belong to that campaign and are not universal constants.Source: §C.2 · p. 11
  • Table 5 is identified as containing registered task contracts, but the supplied passage does not state their fields or values.Source: Table 5 · p. 12

D EXTENDED RESULTS

The extended CICIoT2023 evidence covers all 45 registered activities, while Campaign08 is presented descriptively rather than as thousands of independent replicates. Table 6 distinguishes controller promotion metrics from detector metrics.

D.1 FORMAL V2 AND CAMPAIGN08

  • Table 6 presents extended CICIoT2023 evidence and specifies that its promotion metrics describe the controller, not the detector.Source: Table 6 · p. 12
  • Formal V2 covers all 45 registered activities; Campaign08 aggregates 2,040 nested decisions, which are not 2,040 independent statistical replicates.Source: §D.1 · p. 12

D.2 V14–V18 DOWNSTREAM SEQUENCE

The downstream CICIoT2023 path reports V17’s metric changes relative to V1/RF100, while the PowerDuck case study traces a constrained problem through a paired evaluation and decision.

D.2 V14–V18 DOWNSTREAM SEQUENCE

  • V17 changed macro-F1 by +5.13%, MCC by +8.42%, and detector FPR by −54.0% relative to V1/RF100, while attack recall remained 2.43 pp below baseline.V18 was rejected; this downstream path is separate from formal V2 selection.Source: Figure 3 · p. 12, Table 7 · p. 12

E FULL TRACEABILITY CASE STUDY

  • The PowerDuck contract covers all 15 registered classes with at most ten examples per class under a frozen embedding and split.Its contradiction is broader detection coverage despite limited class-specific evidence.Source: E FULL TRACEABILITY CASE STUDY · p. 12
  • With identical data, split, embeddings, metric code, and seed, the train-only scaling and balanced linear-SVM candidate changed weighted-F1 from 0.9694 to 0.9982.The candidate is retained as a complete paired observation; principle-level causality requires an equal-budget unguided control.Source: E FULL TRACEABILITY CASE STUDY · p. 13

F REPRODUCIBILITY AND EVIDENCE BOUNDARIES

The records support claim-level checking against stated protocols and gate outcomes, but exact reproduction depends on additional artifacts and aggregate results have limits.

F.1 WHAT THE RECORDS SUPPORT

  • The traceability schema, dataset and metric rules, and PowerDuck example let readers compare reported results and controller decisions with protocols and gate outcomes.This supports claim-level checking, not by itself an exact rerun.Source: §F.1 · p. 13
  • Exact reproduction also requires data or retrieval instructions, executable code, a software environment, candidate configuration, and random seeds.When only aggregates are available, claims are limited to reported summaries.Source: §F.1 · p. 13
  • Aggregate results cannot recover missing decision-level data or make related decisions independent experimental repetitions.Source: §F.1 · p. 13

F.2 EVIDENCE-SPECIFIC BOUNDARIES

Formal CICIoT2023 V2 completed 45 activities and improved several metrics, but missed its frozen promotion thresholds; the record also distinguishes campaign decisions from independent repetitions.

F.2 EVIDENCE-SPECIFIC BOUNDARIES

  • Macro-F1 increased from 0.8325 to 0.8483, attack recall increased by 0.018, and detector FPR decreased by 0.0017, but V2 was not promoted.Its relative macro-F1 gain was below 3.2277% and its detector-FPR reduction was below 0.005.Source: §F.2 · p. 13
  • Campaign08 summarizes 2,040 controller decisions within the same experimental structure, so promotion F1, recall, and specificity are descriptive aggregates rather than significance tests.The paper does not calculate p-values from the decision count.Source: §F.2 · p. 13
  • The V17/V18 sequence is a later optimization path, not formal V2 selection; V18 slightly raised attack recall versus V17 but worsened macro-F1, MCC, and detector FPR.V18 remains recorded as a rejected candidate.Source: §F.2 · p. 13
Loading 2610.04555v1…