PaperlayerView exact revision on arXiv

Source-linked AI summary

The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generation

Qi Bing, Xiaowei Shao

arXiv:2608.22956v1cs.CL

TL;DR

The paper studies how to synthesise inference-time concept codes for multi-axis concept-bottleneck text generation. It compares classifier inversion, reference encoding, and a post-hoc label-conditioned prior, finding that inversion underperforms the prior and bare inversion collapses because its code leaves the encoder manifold.

  • Problem

    Inference-time concept-code synthesis is under-specified when concept-bottleneck generation must realise a target attribute configuration without a direct language-model fluency term.

  • Method

    The paper compares bare and regularised classifier inversion, reference-text encoding, and a post-hoc prior fitted to per-combination encoder-code means.

  • Results

    Every tested inversion variant underperforms the prior, while bare classifier inversion collapses to chance from a directly measured off-manifold code.

  • Takeaways & Limitations

    For this concept-bottleneck CTG family, prior inference should be the default code source rather than bare classifier inversion.

  • Takeaways & Limitations

    The collapse claim is limited to concept-bottleneck CTG models whose generators are conditioned through injectors trained on the encoder-code distribution.

Abstract

from arXiv · show

Concept-bottleneck controllable generation routes multi-attribute control through a low-dimensional concept code that, at deployment, must be synthesised from a target attribute configuration. We study this problem in concept-bottleneck text generation under multi-axis compositional generalisation, comparing three ways to obtain the inference-time code: classifier inversion against the encoder heads, reference-text encoding, and a post-hoc label-conditioned prior. Since a concept code admits no direct LM-fluency term, regularising inversion must instead constrain the code toward the encoder's training distribution. We therefore test bare inversion and three regularised variants: label-agnostic and label-conditioned Mahalanobis penalties, and a conditional normalising-flow density baseline. Every inversion variant we test underperforms a simple post-hoc prior fitted to per-combination encoder means on the same checkpoints, across three backbone families spanning $124$M to $8$B parameters. The bare form of classifier inversion also silently collapses to chance, traceable to a directly measured off-manifold code. We validate this diagnosis on real-world benchmarks and under external evaluators, enabling fair comparison with published baselines.

1 Introduction

The paper isolates inference-time concept-code construction in multi-attribute concept-bottleneck generation. It finds that classifier inversion can leave the encoder’s training manifold, whereas a post-hoc label-conditioned prior offers a simpler alternative.

  • Multi-attribute control requires synthesising a concept code from a target configuration at deployment.
  • Classifier inversion lies 3 to 7 times farther from the encoder’s training distribution than a working code.The gap appears in both diagonal-Gaussian Mahalanobis distance and mean distance to 10 nearest training neighbours.
  • The study compares bare and regularised classifier inversion with a post-hoc label-conditioned prior on matched checkpoints.Regularisers include label-agnostic and label-conditioned Mahalanobis penalties and a conditional normalising-flow density baseline.
  • Every tested inversion variant underperforms the prior by 7 to 29 percentage points, while bare inversion collapses to chance.
  • The prior is fitted to per-combination encoder means and recovers compositional generalisation without inference-time optimisation or reference text.The paper also characterises it as a conditional-mean denoiser of sample-specific code variation.

2 Related Work

Prior work has explored interpretable concept bottlenecks, multi-attribute controllable generation, and regularised control-vector optimisation. This paper focuses on the under-specified inference protocol for concept-conditioned generation and extends evaluation to multi-axis compositional generalisation.

  • Concept-bottleneck models constrain outputs through low-dimensional, interpretable codes, while CB-LLMs have primarily evaluated single-axis steering.
  • The paper extends per-axis bottlenecks to multi-axis compositional generalisation on combinations unseen during training.
  • Multi-attribute controllable generation includes joint-training, decoding-time, latent-space, adapter, hidden-state, reinforcement-learning, and feed-forward reweighting methods.
  • Existing classifier-gradient and activation-steering methods commonly add fluency, prototype-distance, or distributional regularisation.
  • Inference-time construction of the concept code remains under-specified in concept-bottleneck generation.The paper isolates this protocol rather than treating it as one component of a compound steering objective.

3 Background

The framework encodes text into continuous per-axis concept vectors, injects the selected code into a frozen Transformer, and requires a separate inference protocol when the encoder path is bypassed. The compared protocols are classifier inversion, reference encoding, and a post-hoc label prior.

  • Problem formulation: Training covers seen attribute combinations, while evaluation reports performance on both seen and unseen combinations.
  • Architecture: The concept encoder produces per-axis sub-codes with classifier heads, concatenated into the full code z.
  • Architecture: The injector maps the encoder code to per-layer gated residual updates for a frozen LoRA-adapted Transformer generator.Training uses teacher-forced language-model, concept-classification, and inter-axis orthogonality losses.
  • Inference protocols: At inference, the encoder path is bypassed, so z⋆ must be synthesised from the target configuration.
  • Inference protocols: CLS-INV optimises the code against classifier heads, whereas REF-ENC encodes one labelled held-out example.
  • Inference protocols: PRIOR fits a post-hoc MLP to encoder-code means for seen configurations and queries it on unseen configurations.Its conditional-mean estimate can average away sample-specific variation, unlike single-sample reference encoding.

4 Experimental Setup

Experiments center on the real-world four-axis Fyelp benchmark, with Amazon and synthetic or single-axis checks. They vary backbone scale and use external benchmark evaluators alongside fluency metrics, with released artefacts for reproducibility.

  • Datasets: The primary Fyelp setting combines sentiment, gender, cuisine, and tense across compositional splits.The study also evaluates Amazon and uses synthetic four-axis and single-axis YelpP checks.
  • Datasets: Fyelp contains 65K training, 1.5K validation, 1.5K seen-test, and 1,750 unseen-test examples.
  • Backbones: The main Fyelp backbones span GPT-2 124M to Qwen-2.5 1.5B, with additional checks reaching 3B and 8B.
  • Evaluation: Evaluation uses the official CompMCTG RoBERTa-large classifier suite rather than the bottleneck’s own classifier heads.Reported metrics include per-axis and mean accuracy, joint all-axes accuracy, perplexity, and Dist-n.
  • Reproducibility: The authors release code, trained components, adapters, and label priors for the four main-table backbones on both splits.

5 Classifier inversion silently collapses across backbones

Classifier inversion collapses across backbones despite healthy training metrics because it drives the concept code off the encoder’s training manifold. Manifold regularisation repairs chance-level failure but does not match the post-hoc prior.

  • Classifier inversion produces near-chance control across trained checkpoints, with 4-axis Fyelp accuracy within 5 pp of the ≈42.5% random baseline.On the seen split, accuracies range from 41.47% to 46.88% across four backbones.
  • Classifier inversion maximises classifier accuracy while training metrics remain healthy, but generated text exhibits repetition loops or off-domain pretrain-style text at generation time.An LLM judge rated none of 20 CLS-INV samples as fluent reviews.
  • The collapse persists on synthetic multi-axis and single-axis YelpP checks across backbone families, injector mechanisms, and parameter scales.Synthetic accuracy stays within ±0.03 of the 0.25 baseline, while YelpP remains at the 0.50 binary baseline.
  • The mechanism: an off-manifold control code: The inverted code is measurably off-manifold: Mahalanobis distance is 3.3–3.7 versus 0.50–0.60 for the prior and approximately 1.0 for REF-ENC.Nearest-neighbour distances show the same separation, directly locating the inversion code outside the encoder distribution.
  • The mechanism: an off-manifold control code: Off-manifold codes over-drive the injector through depth, producing a 40× average over-shoot relative to REF-ENC on LLaMA-3.2 1B and a milder 2.7× over-shoot on GPT-2 124M.On LLaMA-3.2 1B, relative modulation rises from 3.3 at layer 0 to 85 at layer 15.
  • Regularised inversion: Mahalanobis and shell manifold penalties lift inversion from chance, but the best regularised variant still trails the label prior by 7 to 29 pp.The Mahalanobis penalty reaches 57.4% on GPT-2 and 52.8% on LLaMA while reducing perplexity from approximately 130 to approximately 20.

6 A label-conditioned prior recovers compositional control

A post-hoc label-conditioned prior estimates concept codes from target labels and recovers compositional control across backbones. Its conditional-mean objective denoises sentence-specific encoder variation, outperforming reference encoding and regularised inversion.

  • The prior fits per-combination encoder means with a single forward-pass MLP, estimating the conditional mean of the encoder code from target labels.It requires neither inference-time optimisation nor a reference text.
  • 6.1 Prior inference recovers compositional generalisation: The label-conditioned prior recovers 4-axis unseen accuracy on every Fyelp backbone, reaching 55.18% to 76.10% versus classifier inversion near the ≈42.5% random baseline.The reported values are 55.18% for GPT-2 124M, 61.50% for GPT-2-Medium 355M, 64.18% for Qwen-2.5 1.5B, and 76.10% for LLaMA-3.2 1B.
  • 6.1 Prior inference recovers compositional generalisation: On ACD test_unseen, the prior improves over CLS-INV by +7.4 to +18.8 pp and remains 7 to 29 pp ahead of regularised inversion variants.The prior also avoids their 50-step inner optimisation and backbone-specific β sweep.
  • 6.2 Mechanism: prior as a conditional-mean denoiser: The conditional-mean objective averages away variation not shared within a label combination, while inversion lacks a term constraining codes to the training distribution.This objective difference explains why the prior targets a code that remains inside the encoder distribution.
  • 6.2 Mechanism: prior as a conditional-mean denoiser: Within GPT-2-Medium combinations, per-dimension variation is 0.43 versus 0.33 across combination means, so the prior discards larger sample-specific variation while retaining concept signal.Averaging 256 same-combination reference encodings matches the learned prior within 0.4 pp; a conditional flow trails it by 10.5/11.8 pp.
  • 6.2 Mechanism: prior as a conditional-mean denoiser: The additive per-axis code model explains 31.7–47.6% of variance, while independent axes leave residual variation primarily attributed to label-invariant encoder artefacts.This residual is interpreted as dominated by per-sample variation discarded by the prior.

7 Baseline comparison and cross-benchmark robustness

The prior’s advantage persists in matched comparisons and on Amazon and YelpP, while its strongest published-baseline result is cross-scale rather than a state-of-the-art claim. External evaluators and alternative benchmarks reproduce the within-method ordering.

  • 7.1 Comparison with the CompMCTG baselines: In the strictly matched GPT-2-Medium setting, the prior reaches 61.50% Hold-Out and 59.52% ACD unseen accuracy, above the mean of nine published baselines but below the strongest baseline.The contribution is characterised as an inference-protocol correction rather than an accuracy record.
  • 7.1 Comparison with the CompMCTG baselines: The matched prior beats REF-ENC by +12.8 pp on ACD unseen and collapsed classifier inversion by +14–16 pp, while achieving roughly 3× lower perplexity than Meta-CTRL.Its 3-axis ACD unseen accuracy reaches 72.34% despite a near-chance gender axis.
  • 7.1 Comparison with the CompMCTG baselines: On Fyelp Hold-Out, the LLaMA-3.2 1B prior reaches 76.10% 4-axis unseen accuracy, +7.41 pp over Meta-CTRL, but this is a cross-scale comparison.The backbone has roughly 3.5× the parameters of the GPT-2-Medium baselines.
  • Cross-benchmark robustness: On Amazon, PRIOR reaches 76.5% seen accuracy, beating collapsed classifier inversion by +30.0 pp and REF-ENC by +14.4 pp, and is also best on unseen accuracy.The experiment uses the same GPT-2-124M pipeline on a two-axis sentiment × topic dataset.
  • Cross-benchmark robustness: On Amazon’s topic axis, classifier inversion is near chance at 20.2% versus a 16.7% baseline, while the prior reaches 74.3% with a 5× perplexity gap.The binary sentiment axis partly survives, but the content-bearing topic axis collapses cleanly.
  • Cross-benchmark robustness: On matched-backbone single-axis YelpP, classifier inversion remains at the 0.500 binary-chance level while PRIOR reaches 0.964 steerability.PRIOR is +1.4 pp above the 0.95 CB-LLM result under a byte-identical reproduction.

8 Conclusion

The paper diagnoses classifier-inversion collapse in concept-bottleneck controllable generation and recommends prior inference as the default code source. The correction improves reliability but is not presented as a state-of-the-art claim.

  • Classifier inversion collapses to chance on every tested backbone despite healthy training metrics, due to an off-manifold code that over-drives the injector.A manifold-regulariser ablation confirms the diagnosis.
  • A post-hoc label-conditioned prior fitted to per-combination encoder means recovers compositional generalisation on the same checkpoints.The result holds under external evaluators and on a second real-world dataset.
  • At a matched backbone, the prior does not beat the strongest CompMCTG baseline, so the contribution is an inference-protocol correction rather than a state-of-the-art claim.

Limitations

The study’s conclusions are bounded by benchmark granularity, architectural coverage, incomplete mechanistic explanation, and limited scale, language, seed, and regulariser coverage.

  • Benchmark granularity: The Fyelp Hold-Out idx=−0 split holds out one 4-axis combination, making test_unseen coarse and potentially inflating classifier-default artefacts.ACD, with half of all combinations unseen, is reported as the more reliable compositional-shift measurement.
  • Architectural scope: The collapse claim applies to concept-bottleneck CTG models whose generator uses an injector trained on the encoder’s code distribution, not all controllable-generation mechanisms.
  • Mechanistic scope: Distance measurements, activation diagnostics, and regulariser ablations support the off-manifold diagnosis but do not fully explain backbone-dependent activation magnitudes.Injector sensitivity, depth effects, and hidden-state scaling remain for future work.
  • Scale, language, and regulariser coverage: Experiments cover English benchmarks, backbones up to 3B for multi-axis CTG, an 8B single-axis check, and selected regularisers, leaving broader settings untested.Some cells are single-seed, and broader regularisation schemes remain future work.

Ethical considerations

The paper frames its ethical scope around responsible interpretation of benchmark attributes and the limited capability expansion introduced by the work.

  • The Fyelp binary gender label is treated only as a benchmark-specific writing-style marker, not as an identity attribute.The paper does not advocate generating gendered content about identifiable people.
  • The label-conditioned prior improves reliability for already-labelled attributes but does not expand controllability beyond attributes present in training data.Misuse risks therefore remain tied to the underlying controllable-generation setting.

C Second real benchmark: CompMCTG Amazon

The Amazon benchmark tests whether the protocol ordering extends beyond Fyelp. PRIOR remains the strongest protocol, while classifier inversion fails most clearly on the content-bearing topic axis.

  • Amazon is a 2-axis product-review benchmark with sentiment and topic, yielding 12 attribute combinations under an unchanged training and evaluation recipe.The held-out Hold-Out idx=−0 combination is positive_clothing.
  • Classifier inversion has elevated perplexity on Amazon, with PPL 65.1 versus 13.0 for PRIOR on the seen split.The table describes this as a 5× perplexity inflation.
  • PRIOR is the best protocol on both seen and unseen splits, reaching 76.5% 2-axis accuracy on seen data.This improves over classifier inversion by +30.0 pp and over REF-ENC by +14.4 pp.
  • On the six-way topic axis, classifier inversion scores 20.2% on seen and 9.7% on unseen, versus 74.3% and 61.2% for PRIOR.Topic chance accuracy is 16.7%, so the collapse is concentrated on the content-bearing axis.
  • The prior-over-REF-ENC ordering also reproduces on the synthetic 4-axis task across every backbone, consistent with the denoising pattern on natural data.
  • Qualitative generation samples: Qualitative samples show classifier inversion producing a degenerate token-repetition loop, while REF-ENC and PRIOR produce fluent restaurant-review text.The samples come from a GPT-2-124M Fyelp Hold-Out checkpoint.

G LLM-as-judge qualitative evaluation

An external LLM-as-judge study independently evaluates attribute control and fluency because automatic metrics can be distorted by classifier artefacts. The prior leads on both criteria, while classifier inversion produces degraded or repetitive text consistent with off-manifold codes.

  • Metric validation: The automatic-metric caveat motivates the qualitative check because the official classifier has a documented default-prediction artefact on degraded text.The LLM judge provides an independent check rather than replacing the headline automatic metrics.
  • Evaluation setup: The LLM judge rated 20 combination-matched generations per code source for attribute match and fluency under a fixed rubric.The study used GPT-2-124M Fyelp Hold-Out test_seen generations and excluded gender from the policy-controlled axes.
  • Attribute control: 2.45/3 attribute match for PRIOR exceeded REF-ENC at 1.90 and CLS-INV at 0.80, below the ≈1.2/3 chance baseline.The CLS-INV score was below chance because collapsed text often expressed no attribute.
  • Fluency: 20/20 PRIOR generations were fluent, compared with 16/20 for REF-ENC and no fluent reviews for CLS-INV.CLS-INV outputs comprised 9/20 token-salad generations and 11/20 degraded, off-domain generations.
  • Mechanistic interpretation: Classifier inversion codes lie 3–7× farther off-manifold than PRIOR or REF-ENC on both backbones, supporting the proposed collapse diagnosis.The distances use diagonal-Gaussian Mahalanobis distance and mean distance to 10 nearest training neighbours.

M Isolating within-combination denoising: the k-sample control

A within-combination averaging control tests whether the learned prior succeeds by denoising sample-specific encoder variation. Accuracy rises and variability falls as more same-combination references are averaged, approaching the learned prior by k=256.

  • Within-combination averaging: Accuracy increases with the number of same-combination references, while cross-draw variability falls from ±2.0 at k=1 to below ±0.3 for k ≥64.This pattern appears on both Hold-Out and ACD datasets.
  • Convergence to the prior: By k=256, averaged codes reach 68.05 on Hold-Out and 74.45 on ACD, within 0.4 pp of the learned prior.The experiment uses GPT-2-124M and the official 4-axis evaluator.
  • Retrieval connection: The infinite-sample within-combination mean is byte-identical to the nearest-seen-combination retrieval code, linking retrieval and conditional-mean denoising.The measured cosine similarity is 1.000, while retrieval accuracy is 68.8 / 74.6.
  • Interpretation: The prior’s advantage is therefore attributed to averaging sentence-specific variation rather than exploiting measurable label co-occurrence structure.The benchmark reports no measurable label co-occurrence structure for the prior to exploit.
  • Variance decomposition: Within-combination encoder variation exceeds between-combination variation on GPT-2-Medium, with mean per-dimension standard deviations of 0.43 versus 0.33.This establishes a substantial sample-specific component that deterministic label-only predictors cannot recover.
  • Factorised prior: The factorised prior reaches residual MSE 0.20, close to the within-combination noise floor, while explicit interaction terms provide no reliable gains.The interaction model matches the plain prior within noise on GPT-2-Medium and performs worse on LLaMA-3.2 1B.
Loading 2608.22956v1…