PaperlayerView exact revision on arXiv

Source-linked AI summary

Constraint-Guided Enterprise Data Mapping with Large Language Models

Sebastian Monka, Pramod Anantharam, Thien Vo Minh, Lavdim Halilaj

arXiv:2608.24218v1cs.AIcs.CL

TL;DR

Enterprise alignment must accommodate evolving schemas, implicit attributes, and unit or granularity mismatches, while LLM-only matching can violate structural invariants. CGM applies executable constraints before neural ranking and bounded LLM disambiguation, achieving strong validity and transfer across enterprise settings at lower cost and effort. Its evidence also bounds the claim: the synthetic study is a stress test, the expert-effort study is small, and Valentine evaluates column-level rather than record-level transfer.

  • Problem

    Enterprise alignment remains largely manual as heterogeneous schemas, providers, and naming conventions continuously change.

  • Method

    CGM uses schema-grounded executable constraints to restrict candidates, relaxes constraints under noise, then ranks and disambiguates within the feasible set.

  • Results

    CGM transfers across seven enterprise makes with macro F1 0.70, while the hard-admissibility gate raises F1 from 0.08 to 0.66 and a small constrained model matches frontier-model validity at approximately 28× lower cost.

  • Takeaways & Limitations

    Constraints act before neural reasoning, preserving structural validity while retaining semantic flexibility and reducing expert effort by approximately 7×.

  • Takeaways & Limitations

    The synthetic study is a deliberate stress test, the expert-effort study is small, and Valentine tests column-level ranking rather than record-level transfer.

Abstract

from arXiv · show

Enterprise entity alignment must handle semi-structured records, implicit attributes, and unit or granularity mismatches. Manual matching is still common in practice, but does not scale as schemas and providers evolve. LLM-only matching improves semantic recall, yet can violate structural and physical invariants, producing fluent yet operationally invalid correspondences. We propose constraint-guided mapping (CGM), a neuro-symbolic method with three stages: (i) schema-grounded admissibility constraints with metadata mc = <tau_c, delta_c>, where tau_c denotes the constraint type and delta_c provides executable relation and normalization logic; (ii) constraint-restricted candidate generation with cascade relaxation to guarantee a nonempty feasible set under noise; and (iii) neural ranking with bounded LLM disambiguation restricted to that feasible set. Methodologically, constraints operate as hypothesis-space operators rather than post-hoc validators, enabling controlled degradation under relaxation and auditable, human-guidable decisions. On a controlled structural-decoy benchmark, hard admissibility shrinks the candidate space by ~480x without dropping the GT, and a layer-by-layer ablation shows this gate, not the LLM, is the decisive lift (F1 0.08 to 0.66). The benefit is model-independent and adds no extra inference cost: a small model with constraints matches a frontier LLM used without them at ~28x lower cost. The method, not a single tuned configuration, transfers across seven enterprise makes (macro F1 0.70), each under its own automatically discovered, expert-refinable constraints, and lowers expert effort by ~7x versus spreadsheet workflows. Public Valentine results add an external ranking sanity check and mark the boundary: constraints should be hard only where structural invariants are match-determining.

1. Introduction

Enterprise alignment remains largely manual as schemas, providers, and naming conventions evolve. CGM constrains admissible candidates before neural ranking and disambiguation, addressing implicit attributes, unit conversion, and granularity mismatches.

  • Enterprise alignment remains largely manual because schemas, providers, and naming conventions continuously change.
  • CGM formalizes alignment as constraint-guided correspondence inference rather than independently scored schema- and instance-level components.
  • Executable constraints define admissible candidates, while neural components rank and disambiguate only within that bounded hypothesis space.
  • Implicit-attribute extraction, unit conversion, and granularity matching are required in cases where lexical similarity fails.The motivating example converts 1000 cc to 1.0 L while matching differing representations and dates.
  • CGM combines hypothesis-space constraints, cascade relaxation, bounded LLM disambiguation, model-independent validity, and reduced expert effort.The contributions report F1 0.08 →0.66, 100% validity at approximately 28× lower cost, and approximately 7× lower expert effort.

2. Problem Setting and Requirements

Enterprise alignment couples representation, structural, and semantic heterogeneity. The resulting requirements emphasize controlled decomposition, explicit compatibility constraints, graceful degradation, and auditable decisions.

  • Enterprise data combines representation, structure, and semantics heterogeneity across composite fields, units, aggregation, naming, and temporal conventions.
  • Representation and structure define admissibility, while semantics is resolved within admissible candidates.
  • The proposed requirements are controlled decomposition, explicit unit and granularity compatibility, graceful degradation, and auditability.
  • These requirements motivate hard admissibility, cascade relaxation, and bounded neural disambiguation.

3. Related Work

Prior alignment systems trade off invariant enforcement and semantic flexibility. CGM instead applies schema-grounded admissibility before neural reasoning, with relaxation and traceable degradation.

  • Entity matching and alignment are established data-integration problems, with structural-validity enforcement distinguishing major paradigms.
  • Rule systems preserve domain invariants but can be brittle under paraphrases, sparse metadata, and heterogeneous naming.
  • PLM- and LLM-based matchers improve semantic flexibility but may score correspondences without hard admissibility and propagate normalization errors.
  • CGM applies schema-grounded hard admissibility before neural reasoning, uses cascade relaxation, and provides traceable degradation.
  • The evaluation compares CGM with standard and SOTA matchers, an unconstrained retrieval-augmented LLM, and a released synthetic benchmark.

4. Problem Formalization

CGM represents ground-truth alignments as set-valued maps and predictions as bounded shortlists. Hard constraints define an admissible space, relaxation handles noise, and neural ranking plus LLM disambiguation operate within it.

  • The ground-truth alignment maps each source entity to a set of semantically and structurally consistent target entities.
  • Each constraint links source and target attributes through metadata specifying its type and executable relation or normalization logic.
  • Hard constraints eliminate invalid correspondences, whereas soft constraints provide ranking evidence.
  • The admissible space contains targets satisfying every instantiated hard constraint, with relaxation selecting a largest feasible subset under noise.
  • Neural ranking forms a bounded shortlist inside the admissible space, and LLM disambiguation selects only from that shortlist.
  • If all hard constraints are dropped, the admissible space becomes the full target set and the method reduces to unconstrained retrieval.
  • Predictions are complete, partial, or wrong according to their relation to the ground-truth set.

5. Approach

CGM combines executable hard constraints, cascade relaxation, neural ranking, and bounded LLM disambiguation to keep enterprise mappings structurally admissible while preserving semantic flexibility. Expert-editable and automatically discovered constraints support auditable, model-independent mapping across evolving enterprise schemas.

  • Constraint specification: Hard constraints define the admissible candidate set before neural ranking, preserving structural consistency while allowing semantic flexibility.The system applies executable predicates over source-target attributes before neural inference.
  • Constraint specification: Experts specify constraint metadata and executable logic, while the LLM proposes editable predicates that can be validated for safety, schema consistency, and intent alignment.Experts may change transformation logic or switch constraints between hard and soft; failed proposals receive structured feedback and bounded retries.
  • Constraint discovery: Automatic discovery selects hard constraints that maximize recall×selectivity, retaining the gold target while shrinking candidates and using disjoint test data to guard against overfitting.Constraint mining and subset search use a seeded 60/40 train-test split, with reported numbers measured on the test split.
  • Candidate generation: Cascade relaxation handles both empty feasible sets and nonempty but wrong blocks caused by key collisions, reopening candidates when admissible similarity is too low.The method records the active constraint subset during relaxation and reports recovery of colliding-code cases on Make E.
  • Ranking and disambiguation: Within H(es), neural scores combine embedding similarity with soft predicates, and the top-k shortlist is passed to bounded LLM disambiguation rather than the full target set.The LLM selects or abstains only among identifier-bearing candidates already filtered by active hard constraints; post-filtering preserves admissibility.
  • Ranking and disambiguation: Expert priorities become reusable soft prompt artifacts that guide residual partial matches without requiring bespoke code.Priorities can encode ordered attribute preferences and are logged alongside constraints, scores, and relaxation levels.

6. Evaluation

The evaluation tests whether constraints reshape candidate spaces before neural reasoning across synthetic, public, and enterprise settings. Results show decisive gains from hard admissibility, transfer across models and makes, and lower expert effort, with clear scope boundaries.

  • Evaluation design: The evaluation spans a released synthetic ablation, the public Valentine suite, and enterprise data at production noise and scale.The roadmap isolates mechanism, external ranking competitiveness and admissibility boundaries, and deployment generalization.
  • Controlled synthetic ablation: ~480× fewer candidates preserves 100% ground-truth recall under cascade relaxation and leaves every retained candidate structurally valid.The LLM-free gate is evaluated on an approximately 12k-target catalog before neural reasoning.
  • Controlled synthetic ablation: F1 rises from 0.08 to 0.66 when hard admissibility is added, while feature selection and normalization alone do not help.Layer-by-layer deltas attribute the decisive jump to the gate rather than the LLM.
  • Model scale: Every CGM configuration reaches 100% valid-match across model scales, and the smallest constrained model matches the unconstrained frontier model at approximately 28× lower cost.Constraints add no extra LLM calls, so scaling the LLM does not substitute for admissibility.
  • Public Valentine benchmark: On Valentine, CGM reaches MRR 1.00 and Recall@GT 0.71, versus Magneto’s 0.93/0.76, while a rigid type gate lowers MRR to 0.94.The external suite tests column-level ranking and indicates that hard constraints should apply where structural invariants determine matches.
  • Enterprise deployment: Across seven enterprise makes, CGM reaches macro F1 0.70 versus 0.21 for the same naive LLM over raw records.The identical pipeline uses make-specific, automatically discovered constraints, while performance varies with discriminating-key coverage and GT cardinality.
  • Enterprise deployment: Expert effort falls from 10.5 to 1.5 days per make, a roughly 7× reduction with the fewest interactions.The comparison uses median effort across makes for spreadsheet workflows, Rules, Rules + LLM, and CGM.
  • Limitations: The synthetic study is a deliberate stress test rather than a real-world distribution, the expert-effort study is small, and Valentine tests columns rather than records.These limitations bound how broadly the evaluation results should be generalized.

7. Conclusion

CGM places constraints before neural ranking and bounded LLM disambiguation, with reported gains in ablation, transfer, and expert effort.

  • CGM treats constraints as hypothesis-space operators before neural ranking and bounded LLM disambiguation, using cascade relaxation.
  • F1 rises from 0.08 to 0.66 when hard admissibility is added, which the layer-by-layer decomposition identifies as the decisive layer.
  • Macro F1 reaches 0.70 across seven enterprise makes, indicating transfer beyond a single tuned configuration.
  • Expert effort falls approximately 7×, while the method is reported to operate at no extra cost.

Appendix A. Evaluation Roadmap

The evaluation is organized as an evidence ladder: experiments first isolate the constraint mechanism, then test scale, public baselines, transfer, and practical payoff.

  • Evaluation Roadmap: The evaluation roadmap treats each experiment as answering one plain question about the constraint mechanism.The experiments are presented as an evidence ladder rather than unrelated datasets.
  • Evaluation Roadmap: The ladder proceeds from mechanism isolation to model-scale and public-baseline checks, then transfer and practical value.

Appendix B. Synthetic Benchmark: Schema and Worked Example

The synthetic benchmark creates structural mismatches between source and target schemas, then uses executable two-sided constraints to normalize, filter, and rank candidates. A worked example shows that hard admissibility can preserve the ground truth while removing lexically attractive structural decoys.

  • Synthetic Benchmark: Schema: The benchmark uses different source and target schemas, requiring correspondences to cross structural heterogeneity rather than simple name changes.The same four mismatch classes also appear in the enterprise data.
  • Synthetic Benchmark: Worked Example: The ground-truth target is the lexically least similar candidate at 0.34, while every distractor scores 0.75 and violates one hard constraint.This makes similarity- or LLM-only ranking vulnerable to the structural decoys.
  • Synthetic Benchmark: Worked Example: Hard admissibility prunes all three distractors and retains only the ground truth despite their near-identical text.The worked example includes wrong engine code or wrong generation as the violated constraints.
  • Synthetic Benchmark: Rule Engine: A rule consists of source and target expressions over column names, evaluated on every row before execution brings both sides into comparable form.Hard and soft rules share this two-sided structure and differ at execution.
  • Synthetic Benchmark: Rule Engine: Hard execution compares normalized values for equality to gate admissibility, whereas soft execution uses cosine similarity to order survivors.The example converts engine cc to liters and strips the target unit before equality testing; soft naming rules extract and lowercase model names.
  • Synthetic Benchmark: Inference: The candidate list is pre-filtered by hard constraints and model outputs are intersected with that shortlist, making predictions admissible by construction.The prompting setup uses deterministic decoding and supports structured or key-list outputs, with abstention when no confident match exists.

Appendix D. Enterprise Architecture Ablation

The ablation shows that hard admissibility, rather than normalization or the LLM alone, drives the major performance gain, while constraints preserve validity across models and structural conditions.

  • Enterprise Architecture Ablation: F1 rises 0.32→0.60 and complete predictions 7→33 when the hard-admissibility gate is added.Rule-based feature selection and unit/format normalization raise F1 only from 0.21 to 0.32 beforehand.
  • Enterprise Architecture Ablation: 100% of predictions are valid with constraints across frontier, mid, and small chat models, while unconstrained LLM validity remains approximately 10%.The comparison is reported on the full corpus with equal cost and is model-independent.
  • Enterprise Architecture Ablation: Magneto reaches 0% complete and Recall@5 approximately 0.05 because structural decoys and catalog scale prevent reliable target retrieval.At approximately 12k targets, the ground truth appears in Magneto’s top-50 for only 17% of sources.
  • Enterprise Architecture Ablation: Across isolated heterogeneity classes, CGM reaches 42–72% complete at 100% validity, with its largest gains on unit and granularity signals.The unconstrained LLM scores 0–4% complete and 9–15% valid on these adversarial-only instances.

E.4. Extended Synthetic Benchmark

The extended synthetic benchmark tests the same constraint mechanism under deep one-to-many matching and a 9% NO MATCH rate. It reports that the overall picture remains unchanged, while Magneto is limited by retrieval coverage.

  • Extended Synthetic Benchmark: The extended benchmark adds deep one-to-many quotas and a 9% NO MATCH rate to stress the mechanism on a more realistic distribution.All three method families are evaluated with the CGM funnel decomposed in Table 11.
  • Extended Synthetic Benchmark: Magneto records complete 0% and F1 0.02, while its retrieval ceiling Recall@50 0.17 limits the GPT reranker.The reranker can only reorder retrieved top-50 candidates, so retrieval coverage bounds its result.

Appendix F. Enterprise Coverage and Deployment Details

Enterprise deployment evaluates automatically discovered, expert-validated constraints on held-out per-make data, including coverage, transfer, and expert effort. The evidence emphasizes that constraint primitives transfer across makes and that discriminating rules can sharply reduce candidate spaces while preserving recall.

  • Enterprise Coverage and Deployment: The enterprise deployment uses the same automatically discovered constraints for per-make quality, coverage ablation, and expert-effort evaluation.This reuse underscores transfer of the constraint primitives across makes.
  • Enterprise Coverage and Deployment: The enterprise evidence is organized around an LLM-free rule-ablation table and a full expert-effort study on the internal held-out split.
  • Enterprise Coverage and Deployment: Constraint selection and evaluation are separated by a fixed 60/40 train/test split for each make.Candidate mining and subset search use training data, while the end-to-end pipeline and overfit guard use held-out test data.
  • Enterprise Coverage and Deployment: The hard gate is the decisive jump in the LLM/CGM ladder, while Magneto is reported at retrieval because Recall@50 caps its reranker.The ladder uses gpt-5.4-mini on 200 held-out examples; Magneto uses an mpnet retriever with three seeds and N=500.
  • Enterprise Coverage and Deployment: The evaluation reports held-out test-split numbers because neither the selector nor the LLM sees evaluation data.Train and test metrics, along with keep, discard, and overfit decisions, are journaled per make.
  • Enterprise Coverage and Deployment: The auto-selected configuration collapses the candidate space at near-flat recall, while removing the discriminating identifier rule re-inflates it.The enterprise ablation measures recall and selectivity, with selectivity defined as 1 − |H|/|E_t|.
Loading 2608.24218v1…