PaperlayerView exact revision on arXiv

Source-linked AI summary

Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressions

Yifan Lu, Qiyue Zhang, Haotian Shan, Hanjie Chen, Jiarong Xing

arXiv:2609.34326v1cs.LG

TL;DR

LLM routers commonly use neural query embeddings, yet scaling encoder size brings little routing improvement. REGEXROUTE discovers interpretable regex features from SAE latents and uses them for routing without neural encoding at inference. Across four benchmarks, 128 fixed features achieve 76.43% average routing accuracy, comparable to neural-encoder routers.

  • Problem

    Although larger neural encoders are expected to capture richer query intent and difficulty, Qwen2.5 scaling from 0.5B to 72B brings little routing improvement.

    Source: §1 · pp. 1–2

  • Method

    REGEXROUTE uses SAE descriptions and activation patterns with an LLM to discover and refine regex features from unlabeled text for inference without an LM or SAE forward pass.

    Source: §1 · p. 2

  • Results

    76.43% average routing accuracy across four benchmarks with 128 regex features is comparable to 74.92–76.41% for neural-encoder routers.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings establish compact explicit surface features as sufficient for competitive routing and offer a possible direction for designing and understanding LLM routers.

    Source: §1 · p. 2, §8 · p. 10

    Feature discovery uses general unlabeled web text kept separate from routing benchmark data and evaluation results.

    Source: §3 · p. 3

Abstract

from arXiv · show

Large Language Model (LLM) routers commonly rely on neural query embeddings, with larger encoders expected to better capture query intent and difficulty. Yet scaling Qwen2.5 encoders from 0.5B to 72B parameters brings little improvement in routing accuracy (Figure 1b), suggesting that small encoders may already capture the query properties needed for routing. We therefore investigate which properties matter and whether they can be extracted directly from text without a neural encoder. We introduce REGEXROUTE, a pipeline that uses sparse autoencoders (SAEs) to discover interpretable regular-expression (regex) features. Using unlabeled text, an LLM turns descriptions of grouped SAE latents into regex extractors and refines them to match latent activation patterns. These extractors supply numerical features to a lightweight routing head, eliminating neural encoding at inference (Figure 1a). Across four benchmarks, one fixed set of 128 features achieves 76.43% average routing accuracy, comparable to 76.41% for the strongest neural text encoder baseline, with much smaller latency and strong robustness. These findings establish explicit, interpretable text features as a practical basis for designing and understanding LLM routers.

1 INTRODUCTION

LLM routers commonly map queries to neural embeddings, but increasing encoder size from 0.5B to 72B brings little routing improvement. REGEXROUTE discovers regex features from SAE latents and achieves comparable accuracy across four benchmarks without neural encoding at inference.

1 INTRODUCTION

  • Scaling Qwen2.5 encoders from 0.5B to 72B parameters brings little routing improvement, motivating a search for query properties extractable without neural encoding.The paper investigates whether small encoders already capture the properties routing needs.Source: §1 · pp. 1–2
  • REGEXROUTE uses SAE latent descriptions and activation patterns to guide an LLM in authoring and refining interpretable regex feature extractors.The extractors are discovered from unlabeled text.Source: §1 · p. 2
  • 76.43% average routing accuracy across four benchmarks with 128 regex features is comparable to 74.92–76.41% for neural-encoder routers.The same features are intended to replace neural text encoders for routing.Source: §1 · p. 2
  • The authors propose explicit, interpretable text features as a direction for designing and understanding LLM routers.Each regex feature measures a query property directly from its text.Source: §1 · p. 2

2 RELATED WORK

Prior routing approaches use neural text encoders, while interpretable approaches derive query concepts in several ways. REGEXROUTE distinguishes itself by discovering regex features from SAE latents without routing labels or a predefined task schema.

2 RELATED WORK

  • Difficulty-aware, preference-aligned, and clustering-based routers all use neural text encoders to represent queries.These approaches select models based on query difficulty, human preferences, or cluster-level cost-effectiveness, respectively.Source: §2 · p. 2
  • Interpretable routers use predefined query concepts or generated tags, with properties specified by theory, language models, or labeled downstream data.Their property values are typically computed by prompting an LM or probing a neural embedding at inference.Source: §2 · p. 2
  • REGEXROUTE discovers regex features from SAE latents, unlike prior interpretable approaches that use predefined concepts or labeled data.Its feature discovery uses unlabeled text without routing labels or a predefined task schema.Source: §2 · p. 2

3 PRELIMINARIES

The preliminaries define the routing dataset and its train, validation, and test splits, and describe how unlabeled text supports feature discovery. They also define SAE activations at token and document levels.

3 PRELIMINARIES

  • The routing dataset pairs each query with scores for candidate models; disjoint training, validation, and test splits support fitting, hyperparameter selection, and evaluation.A router maps a query to a model index.Source: §3 · p. 3
  • Feature discovery uses a general unlabeled web-text corpus separate from routing benchmark data and evaluation results.Fit and held-out portions are used to refine and select regex extractors, then evaluate alignment with SAE activations.Source: §3 · p. 3
  • An SAE maps a language-model hidden activation at a token position to a sparse vector of latent activations, which can be summed across document tokens.Each latent has a human-readable description, and its document-level activation is obtained by summing over token positions.Source: §3 · p. 3

4 METHOD

REGEXROUTE discovers reusable regex features by grouping SAE latents, authoring extractors from latent descriptions and associated text, and refining them against activation patterns. It freezes the extractors and routes queries using their numerical features.

4.1 DISCOVERING REGEX FEATURE EXTRACTORS

  • Related SAE latents are grouped into families because individual latents may be too narrow and per-latent extractor construction would be costly.Latents are associated by absolute correlation across documents and clustered with normalized spectral clustering, which limits maximum family size.Source: §4.1 · p. 3
  • For each family, an LLM uses latent descriptions and activation-associated tokens to propose regexes that detect reusable surface patterns and aggregate matches into scalar features.Candidate extractors include a human-readable name, a regex, and an aggregation rule such as match count or proportion.Source: §4.1 · p. 4
  • Regex candidates are refined against family activation patterns, ranked by Kendall’s τb alignment, and the highest-alignment extractor for each family is frozen.The alignment score measures how faithfully regex outputs preserve document ordering.Source: §4.1 · p. 4

4.2 ROUTING WITH THE DISCOVERED REGEX EXTRACTORS

  • A trained Extra-Trees regressor predicts candidate-model response scores from the frozen regex features and selects the model with the highest predicted score.The routing head is trained on the routing dataset after the regex library is frozen.Source: §4.2 · pp. 4–5
  • At inference, frozen regex extractors turn each query into numerical features for model selection without an LM or SAE forward pass.The pipeline overview separates offline feature discovery from inference with the fixed extractor set.Source: §4 · p. 3, Figure 2 · p. 4, §4.2 · p. 5

5 EXPERIMENTS

Across four benchmarks, 128 regex features achieve mean routing accuracy comparable to the strongest embedding baseline. The experiments also find the highest mean accuracy under query perturbations and lower median routing latency on CPU than a neural encoder router on GPU.

5.1 EXPERIMENTAL SETUP

  • Evaluation covers four benchmarks, comparing Extra-Trees routing heads over neural encoder, sentence-embedding, and TF-IDF representations.Hyperparameters are selected on validation data, and results report each benchmark and their unweighted mean.Source: §5.1 · p. 5

5.2 ROUTING QUALITY WITHOUT EMBEDDINGS

  • 76.43% mean routing accuracy with 128 regex features is comparable to 76.41% for the strongest embedding baseline (Table 1).The reported accuracy is the unweighted average across four benchmarks.Source: §5.2 · p. 5, Table 1 · p. 6

5.3 ROBUSTNESS TO QUERY PERTURBATIONS

  • 75.62% mean perturbed accuracy is highest; regex accuracy drops similarly to Qwen2.5-0.5B, while sentence-embedding models score lower despite losing less accuracy.Perturbations include misspellings, word substitutions, sentence restructuring, and paraphrasing; methods are fitted on clean text.Source: §5.3 · pp. 5, 7, Figure 3 · p. 6

5.4 ROUTING LATENCY

  • 2.2 ms median latency on CPU versus 19.5 ms for Qwen2.5-0.5B on GPU; CPU routing leaves GPU resources available for response generation.Latency is measured from query input to model selection, including feature extraction and routing-head inference.Source: Figure 3 · p. 6, §5.4 · p. 7

6 ABLATION STUDIES

The ablations examine discovery models, feature count, latent grouping, feature authoring and refinement, and routing heads. They find that several configurations retain competitive accuracy, while SAE guidance, alignment-based refinement, and tree-based heads improve the suitability of regex features for routing.

6.1 CHOICE OF DISCOVERY LM AND SAE

  • Qwen and Llama discovery configurations reach 76.44% and 76.37% accuracy, respectively, compared with 76.43% for the Gemma configuration.The results show that the pipeline can produce general regex features from multiple evaluated LM–SAE pairs.Source: §6.1 · p. 7

6.2 NUMBER OF REGEX FEATURES

  • Accuracy rises by 2.6 percentage points from N = 1 to 32, then saturates between N = 32 and 128, making 32 features a lower-latency alternative.The authors retain N = 128 as the main configuration; further expansion primarily increases extraction cost.Source: §6.2 · p. 7

6.3 CHOICE OF LATENT GROUPING

  • At N ≥ 64, mean accuracy differs by at most 0.40 percentage points across four grouping methods; spectral clustering also yields the smallest largest family at N = 128.The largest spectral-clustering family contains 439 latents, compared with 531 to 4,904 for the alternatives; the authors retain it to limit the latents each extractor must approximate.Source: §6.3 · p. 8

6.4 DO SAE GUIDANCE AND REFINEMENT IMPROVE REGEX FEATURES FOR ROUTING?

  • At N = 32 and 128, SAE guidance raises test accuracy over LLM-only authoring by 1.00 and 0.49 percentage points, and refinement adds 0.38 and 0.27 points, respectively.The authors interpret these gains as evidence that latent descriptions and associated tokens guide feature authoring, while alignment feedback helps regexes approximate SAE activation patterns.Source: §6.4 · p. 8

6.5 CHOICE OF ROUTING HEAD

  • Tree-based heads achieve the highest mean accuracy, whereas k-NN and k-means perform less well with regex features.The authors attribute this pattern to tree heads learning thresholds on individual count- and ratio-based coordinates, unlike fixed distances across distinct properties.Source: §6.5 · p. 8

7 ANALYSIS

The analysis finds that regex coordinates expose interpretable differences among query sources and group queries by source and model performance. It also finds that small encoders already recover many of these surface features, consistent with the limited routing gains from scaling encoders.

7.1 INTERPRETABLE SURFACE FEATURES DISTINGUISH QUERIES

  • MMLUPro is elevated on short-line and numeric features, XQuAD on script and accent features, and NarrativeQA on prose word counts.The 128 features are grouped into eight surface-property categories: script, casing, prose, layout, numbers, paths, math, and code.Source: §7.1 · p. 9
  • Regex features group queries by source and observed model score about as well as Qwen2.5-0.5B and better than E5-base and ModernBERT-base.Both measures significantly exceed the random baseline, using neighborhoods of 20 test queries that share the most Extra-Trees leaves.Source: §7.1 · p. 9

7.2 SMALL ENCODERS ALREADY CAPTURE SURFACE FEATURES

  • With an MLP probe, Qwen2.5-0.5B reaches R2 = 0.820, while the roughly 147× larger 72B model reaches 0.848.Linear recovery improves from 0.612 to 0.751; the authors conclude that much surface information is already recoverable from the small encoder through a nonlinear mapping.Source: §7.2 · p. 9, Figure 6 · p. 10

8 CONCLUSION

The conclusion presents REGEXROUTE as a way to discover interpretable regex features from SAE descriptions and activations, with a fixed 128-feature library achieving accuracy comparable to neural embeddings. The accompanying appendices detail authoring prompts, benchmark cleanup, discovery data and models, and the encoder-scaling comparison.

8 CONCLUSION

  • A fixed library of 128 regex extractors achieves routing accuracy comparable to neural embeddings across four benchmarks, with low CPU latency and competitive performance under query perturbations.The conclusion also reports that SAE guidance and activation-based refinement improve routing performance, and that the pipeline works across the evaluated discovery model families.Source: §8 · p. 10

A.1 SAE-GUIDED INITIAL AUTHORING; A.3 AUTHORING WITHOUT SAE GUIDANCE

  • The SAE-guided initial prompt uses latent-family descriptions and associated tokens to propose surface-pattern extractors; the LLM-only control instead authors without SAE evidence or alignment feedback.The prompt instructions favor symbols, orthography, and structure rather than English content-word literals or topical vocabularies.Source: §A.1 · pp. 14–15, §A.3 · p. 16

A.2 ALIGNMENT-BASED REFINEMENT

  • The refinement prompt uses fit-set alignment feedback to request revised or alternative extractors, after which the controller retains the fit-best candidate from six proposals per family.The candidates are scored by Kendall tau-b on fit documents.Source: §A.2 · pp. 15–16

B.1 BENCHMARKS TEMPLATE REMOVAL

  • Benchmark templates are removed before any representation sees the queries, while original response scores are retained.The procedure removes source-specific wrappers and labels, standardizes query layout, and applies the same cleaning to all splits and perturbed test queries.Source: §B.1 · p. 17

B.2 PIPELINE CONFIGURATION

  • The unlabeled discovery corpus contains 20,035 public documents spanning web prose, mathematics, code, and instruction/dialogue text.The split uses 13,394 fit documents and 6,641 held-out documents; the same split is used across discovery models.Source: §B.2 · p. 18
  • The default discovery setup uses frozen Gemma-2-2B and its 16,384-latent SAE; evaluated alternatives include Qwen3-1.7B and Llama-3.1-8B with pretrained SAEs.The SAEs encode the residual stream after the final transformer block, and their activations supply latent families and alignment targets.Source: §B.2 · p. 18

B.3 QUERY REPRESENTATIONS

  • For the scaling study, every query is encoded with Qwen2.5 base models using a prefill-only bf16 forward pass, last-token pooling, and an 8,192-token context with head truncation.Table 4 lists neural encoders by category, with their checkpoints, dimensions, pooling methods, and supported context lengths.Source: §B.3 · p. 18

B.4 ENCODER SCALING SETUP

  • Despite a 147× increase in Qwen2.5 model size, mean test accuracy ranges from 76.31% to 76.52%; the 128 regex features reach 76.41% without neural encoding at inference.The reported value is the unweighted mean across four benchmarks; all representations follow the same evaluation protocol.Source: §B.4 · pp. 18–19

B.5 ROBUSTNESS DETAILS

The robustness details evaluate rewritten and mechanically corrupted queries, while the latency details report per-query costs and traffic-weighted tail behavior. Regex routing remains the most accurate evaluated representation under character corruption, but the 128-feature router has a tail-latency trade-off on realistic query lengths.

B.5 ROBUSTNESS DETAILS

  • At 30% character corruption, regex accuracy falls 1.43 percentage points from clean, versus 0.96–1.16 points for sentence-embedding models, yet remains highest among evaluated representations at every corruption level.The study also tests misspellings, word substitutions, sentence restructuring, and paraphrasing; one rewrite is generated per test query and mode.Source: §B.5 · pp. 19–20, Table 6 · p. 20

B.6 LATENCY DETAILS

  • At N = 128, one query costs 3.14 ms for regex extraction, 1.05 ms for forest prediction, and 0.01 ms for standardization; extraction time grows linearly with N.On the traffic-weighted query-length distribution, the 128-feature router's p99 exceeds GPU baselines; the 32-feature router stays below all but MiniLM-L6 and E5-base.Source: §B.6 · p. 20, Table 8 · p. 20, Table 7 · p. 20

C ABLATION DETAILS

This section documents ablation choices and routing components, including discovery-model and latent-grouping choices, SAE guidance and refinement, the routing head, and the frozen regex feature library. It also lists regex features for mathematical notation and code and reports comparisons across discovery-model–SAE pairs and grouping methods.

C.1 CHOICE OF DISCOVERY LM AND SAE

  • Routing accuracy is similar across evaluated discovery LM–SAE pairs, with per-benchmark scores and SAE dictionary sizes reported for N = 128.The pretrained SAE checkpoints and discovery corpus are described in Appendix B.2.Source: Table 9 · p. 21, §C.1 · p. 21

C.2 CHOICE OF LATENT GROUPING

  • The grouping comparison applies spectral clustering, spherical k-means, facility-location selection, and average-linkage clustering to the same 16,384 Gemma SAE latents using 13,394 fit documents.Each method receives the same authoring budget and alignment feedback; its regex library is evaluated with an Extra-Trees routing head.Source: §C.2 · p. 21
  • At N = 128, spectral clustering produces the smallest maximum latent family; Table 10 compares mean routing accuracy across four benchmarks for the grouping methods and feature counts.The authors prefer smaller maximum families because each is approximated by one extractor, while a very large family may combine properties that one surface pattern cannot track.Source: Table 10 · p. 21, §C.2 · p. 21, Table 11 · p. 22

C.3 SAE GUIDANCE AND REFINEMENT

  • The SAE-guidance ablation compares an LLM-only control, SAE guidance without refinement, and full feedback-driven revision under a shared six-candidate budget per feature.Both guided variants use the same Gemma-2-2B SAE partition and family briefs, and select among six extractors using Kendall τb on the unlabeled fit corpus; full refinement returns initial candidates and fit scores before revision.Source: §C.3 · p. 22

C.4 ROUTING-HEAD DETAILS

  • For routing-head comparisons, the main head is an Extra-Trees regressor trained on the full per-model score vector, with at least 25 samples per leaf and hyperparameters selected on validation.The Section 6.5 ablation also evaluates a decision tree, random forest, cosine k-nearest neighbors, and mini-batch k-means.Source: §C.4 · p. 22

D COMPLETE REGEX FEATURE LIST

  • The frozen library has 128 regex features across eight surface-property categories, distinct from the SAE latent families used during discovery; it comprises 124 count and four ratio features.Count features use non-overlapping regex matches; ratio features express a count per approximate word, using five characters per word.Source: D COMPLETE REGEX FEATURE LIST · pp. 22–23
  • The math features count LaTeX commands, symbols, numbers, units, and other mathematical notation.The listed patterns include counts for TeX commands and brackets, acronyms or units, LaTeX density, and numbers inside or outside dollar-delimited expressions.Source: D COMPLETE REGEX FEATURE LIST · p. 29
  • The code features count language-agnostic syntax and idioms alongside indicators for Python, C, and Java code.Patterns cover punctuation, keywords, capitalization, code-like identifiers, and language-specific constructs.Source: D COMPLETE REGEX FEATURE LIST · pp. 29–30
  • Additional code patterns detect Python and Java structures, including imports, definitions, indentation, and language-specific keywords.The listed features also include broad header and module patterns and Java access modifiers and statement endings.Source: D COMPLETE REGEX FEATURE LIST · p. 30
  • Other features count Python module and keyword patterns, PHP sigils, migration-like syntax, and machine-learning code idioms.Their patterns include double-underscore names, PHP operators and tags, indentation and literals, dotted calls, and bracketed indexing or slicing.Source: D COMPLETE REGEX FEATURE LIST · pp. 30–31
  • The list also includes features for docstring and script structure, web code, comments, and package-like patterns.These patterns detect cues such as docstrings, decorators, tags, script imports, URLs, quoted strings, and indentation.Source: D COMPLETE REGEX FEATURE LIST · p. 31
  • A further feature counts Python object-oriented keywords such as self, class, super, and raise.The pattern also includes return, import, exception-handling, and boolean or null-value keywords.Source: D COMPLETE REGEX FEATURE LIST · p. 31

E SOURCE DATASET FEATURE PROFILES

The regex coordinates show distinct average feature profiles across source datasets, offering interpretable cues for distinguishing their queries. The accompanying analysis defines standardized source-level profiles and describes how query neighborhoods and similarity metrics are constructed.

E SOURCE DATASET FEATURE PROFILES

  • PIQA peaks on short-line counts, MMLU professional law on word and long-word counts, MATH on LaTeX and mathematical-symbol features, and MuSR on sentence-boundary and punctuation counts.Bars show each source’s mean feature value after standardizing query values with training statistics and clipping to [−4, 4].Source: E SOURCE DATASET FEATURE PROFILES · p. 32, Figure 9 · p. 32, Figure 10 · p. 33

F.1 SETUP AND DEFINITIONS

  • The analysis selects k = 20 test-query neighbors using shared Extra-Trees leaves or standardized Euclidean distance, then evaluates source agreement and model-score similarity.Forest neighbors are ranked by the fraction of trees sharing a terminal leaf; metrics are averaged across the same neighbors and test queries.Source: §F.1 · p. 35

F.2 NEAREST-NEIGHBOR RESULTS

Regex features produce neighborhoods with source and model-score similarity comparable to Qwen2.5-0.5B under shared forest leaves, but weaker direct Euclidean neighborhoods. The accompanying recovery analysis setup probes how well frozen embeddings predict the regex feature values.

F.2 NEAREST-NEIGHBOR RESULTS

  • 68.24% mean source agreement and 64.97% model-score similarity for REGEXROUTE compare with 68.09% and 65.48% for Qwen2.5-0.5B in forest neighborhoods.Both REGEXROUTE means exceed E5-base, ModernBERT-base, and the random reference.Source: §F.2 · p. 35, Table 14 · p. 36
  • 67.00% source agreement and 64.62% model-score similarity for REGEXROUTE compare with 72.27% and 65.62% for Qwen2.5-0.5B under direct Euclidean neighborhoods.Regex features still group queries by source and model performance under this definition, although Qwen has stronger direct neighborhoods.Source: §F.2 · p. 35, Table 15 · p. 36, Table 16 · p. 36

G.1 SETUP AND DEFINITIONS

  • Ridge and MLP probes predict regex feature values from frozen embeddings; recovery summaries include importance-weighted, mean, and median feature R2 across benchmarks.Importance weights come from Extra-Trees feature importances, while mean and median treat features equally.Source: §G.1 · p. 37

G.2 LINEAR AND NONLINEAR RECOVERY RESULTS

Feature recovery is already strong with Qwen2.5-0.5B, and increasing encoder size brings only a small gain for MLP probes but a larger gain for ridge regression.

G.2 LINEAR AND NONLINEAR RECOVERY RESULTS

  • Importance-weighted R2 rises from 0.820 to 0.848 with an MLP and from 0.612 to 0.751 with ridge, comparing Qwen2.5-0.5B with 72B.Mean feature R2 for the MLP rises from 0.734 to 0.761 without importance weighting.Source: Table 17 · p. 37, §G.2 · p. 37, Table 18 · p. 38
Loading 2609.34326v1…