PaperlayerView exact revision on arXiv

Source-linked AI summary

GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction

Ihor Stepanov, Oleksandr Lukashov, Mykhailo Shtopko, Vivek Kalyanarangan

arXiv:2605.10108v1cs.CLcs.LG

TL;DR

NER and RE are often separated, allowing recognition errors to cascade and leaving no unified model for flexible joint extraction. GLiNER-Relex combines both tasks with shared representations and natural-language labels, achieving competitive zero-shot results across benchmarks while retaining encoder efficiency. Its main boundaries are weaker performance than fully supervised models or frontier LLMs on some benchmarks and difficulty with dense entity graphs and many fine-grained relation types.

  • Problem

    Pipeline NER and RE systems can propagate errors, while existing GLiNER-family approaches do not jointly identify entities and extract relations in one shared forward pass.

  • Method

    GLiNER-Relex uses a unified encoder, dedicated relation scoring, and natural-language entity and relation labels to jointly extract entities and relation triplets.

  • Results

    GLiNER-Relex achieves competitive zero-shot performance across four RE benchmarks, including 31.3% on DocRED versus 18.6% for GPT-5-mini and 12.4% for GLiNER2.

  • Takeaways & Limitations

    The model offers efficient, retraining-free extraction of arbitrary entity and relation types through a simple inference API.

  • Takeaways & Limitations

    Performance does not yet match fully supervised models or frontier LLMs on all benchmarks, and dense entity graphs increase spurious predictions under quadratic pair enumeration.

Abstract

from arXiv · show

Joint named entity recognition (NER) and relation extraction (RE) is a fundamental task in natural language processing for constructing knowledge graphs from unstructured text. While recent approaches treat NER and RE as separate tasks requiring distinct models, we introduce GLiNER-Relex, a unified architecture that extends the GLiNER framework to perform both entity recognition and relation extraction in a single model. Our approach leverages a shared bidirectional transformer encoder to jointly represent text, entity type labels, and relation type labels, enabling zero-shot extraction of arbitrary entity and relation types specified at inference time. GLiNER-Relex constructs entity pair representations from recognized spans and scores them against relation type embeddings using a dedicated relation scoring module. We evaluate our model on four standard relation extraction benchmarks: CoNLL04, DocRED, FewRel, and CrossRE, and demonstrate competitive performance against both specialized relation extraction models and large language models, while maintaining the computational efficiency characteristic of the GLiNER family. The model is released as an open-source Python package with a simple inference API that allows users to specify arbitrary entity and relation type labels at inference time and obtain both entities and relation triplets in a single call. All models and code are publicly available.

1 Introduction

GLiNER-Relex addresses the limitations of pipeline NER and RE by jointly recognizing entities and extracting relations in one unified model. It supports zero-shot entity and relation types, benchmarks four RE datasets, and provides an open-source API.

  • Pipeline NER-to-RE systems propagate entity-recognition errors into relation extraction, motivating joint modeling of both tasks.
  • GLiNER-Relex extends GLiNER with a dedicated relation extraction module for joint NER and RE within a single encoder.
  • The model supports arbitrary entity and relation types specified through natural-language labels at inference time.
  • Its relation scoring module is inspired by knowledge graph embedding approaches.
  • Evaluation covers CoNLL04, DocRED, FewRel, and CrossRE, with comparisons against GLiREL, GLiNER2, and GPT-5-mini.
  • The released model provides a straightforward Python API through the GLiNER package.

2 Related Work

Related work spans fixed-type and zero-shot NER, pipeline and joint RE, and increasingly efficient encoder-based and language-model approaches. GLiNER-Relex targets the underexplored intersection of efficient encoding and genuinely joint, flexible extraction.

  • Named Entity Recognition: Supervised NER models use predefined entity types, whereas zero-shot methods recognize types specified at inference time.
  • Named Entity Recognition: GLiNER formulates NER as matching text spans with natural-language entity-type descriptions in a shared bidirectional encoder.
  • Relation Extraction: Pipeline RE identifies entities before classifying their relations, but errors can propagate between these stages.
  • Relation Extraction: Joint RE models entity recognition and relation extraction simultaneously using sequence-labeling and decomposition-based paradigms.
  • Zero-Shot Relation Extraction: Zero-shot RE includes entailment, embedding-learning, multiple-choice, template-based, prompt-based, and generative approaches.
  • Zero-Shot Relation Extraction: LLM-based RE methods use in-context learning, fine-tuning, or synthetic-data generation for sentence- and document-level extraction.
  • Efficient Joint Extraction: GLiREL improves efficient relation classification but requires pre-identified entities, while other efficient joint methods remain fixed-type or supervised.

3 Method

GLiNER-Relex combines a shared encoder, span-based entity extraction, entity-pair construction, relation scoring, and multi-task training. Its unified input jointly contextualizes text with user-specified entity and relation labels.

  • Architecture: GLiNER-Relex accepts text plus user-specified entity and relation labels and outputs typed entity spans and relation triplets.
  • Architecture: The architecture contains a shared encoder, span representation layer, entity-pair construction module, relation scoring layer, and multi-task objective.
  • Input Representation: The unified input concatenates entity-type labels, relation-type labels, and the input sentence in one sequence.
  • Input Representation: [ENT] and [REL] delimiters place type labels in the encoder’s shared context, whose hidden states become type embeddings for downstream scoring.
  • Notation: The notation distinguishes inference-time label sets Y_E and Y_R from the recognized entity-span set E.

3.3 Shared Encoder

The shared bidirectional encoder contextualizes text tokens together with entity and relation labels. Its hidden states provide word, entity-type, and relation-type representations used for span and relation scoring.

  • A bidirectional transformer encoder processes the unified sequence, using DeBERTa-v3 in the implementation.
  • The encoder produces word, entity-type, and relation-type representation sets from contextualized hidden states.
  • Word embeddings aggregate subword token embeddings and may receive additional bidirectional LSTM sequence modeling.
  • Candidate span representations combine start and end token representations with learned width embeddings.
  • Entity scores use dot-product similarity between span representations and projected entity-type representations.
  • Greedy span selection with confidence threshold τ_E decodes the recognized entities.

3.5 Entity Pair Construction

GLiNER-Relex constructs candidate entity pairs after recognition using either exhaustive enumeration or optional adjacency-guided selection. The released checkpoint uses exhaustive all-pairs enumeration, while adjacency decoders remain configurable framework options.

  • Entity Pair Construction: The framework offers all-pairs enumeration and optional adjacency-guided selection for constructing candidate entity pairs.Adjacency-guided selection predicts a soft adjacency matrix and can reduce candidates before relation scoring.
  • All-pairs enumeration: Given |E| recognized entities, all-pairs enumeration creates |E| × (|E| −1) ordered candidate pairs and scales quadratically.This exhaustive strategy is best suited to sentences containing a moderate number of entities.
  • Adjacency-guided selection: Adjacency-guided selection predicts a masked soft adjacency matrix over entity span representations before retaining likely pairs.The mask removes entries involving padded entities, and retained pairs pass a threshold before relation classification.
  • Adjacency-guided selection: The adjacency path supports dot-product, bilinear, MLP, attention, and GCN decoder architectures.These options range from parameter-free similarity to nonlinear, attention-based, and graph-convolutional scoring.
  • Released checkpoint: The released checkpoint uses all-pairs enumeration and activates no adjacency decoder, leaving systematic decoder ablation for future work.Recognized entity span representations are then used for downstream relation scoring.

3.6 Relation Scoring

Relation scoring represents recognized entity pairs and compares them with candidate relation embeddings produced by the shared encoder. The released model uses an MLP pair representation layer because it offers the best reported balance of accuracy and efficiency.

  • Relation Scoring: The model scores selected entity pairs against candidate relation types using shared-encoder relation embeddings.Head and tail span representations are extracted for each pair, while relation type embeddings are jointly produced by the encoder.
  • Pair representation layer: The pair representation layer concatenates head and tail representations and projects them through an MLP into the shared embedding dimension.The resulting pair representation is compared with the relation type embedding by a dot product.
  • Pair representation layer: Dot-product scoring places entity-pair representations and relation type embeddings in a shared semantic space.This formulation supports zero-shot generalization to unseen relation types specified through natural-language descriptions at inference time.
  • Triple scoring layers: The framework also supports knowledge graph–inspired triple scoring functions that model interactions among head, relation, and tail representations.All variants produce scalar compatibility scores over entity-pair and relation-type combinations in a batched operation.
  • Released model: The MLP pair representation layer achieves the best reported balance of accuracy and efficiency and is used in the released model.Knowledge graph–inspired alternatives may help applications requiring symmetry, transitivity, or compositionality priors.

3.7 Training Objective

Training combines entity extraction and relation classification losses, with an optional adjacency loss when adjacency-guided pair selection is enabled. Focal loss and optional negative sampling address class imbalance in both tasks.

  • Training Objective: The multi-task objective combines entity extraction, optional adjacency, and relation classification losses.The adjacency component is included only when adjacency-guided pair selection is used, and coefficients control each loss contribution.
  • Loss components: The loss coefficients λE, λA, and λR determine the relative contributions of entity, adjacency, and relation losses.The adjacency coefficient is therefore relevant only for configurations that activate the adjacency path.
  • Class imbalance: All losses use focal loss with optional negative sampling to handle severe class imbalance in entity and relation extraction.When γ = 0, focal loss reduces to α-balanced binary cross-entropy.
  • Focal loss: The focal-loss formulation exposes α as a balancing factor and γ as the focusing parameter.The training interface can increase γ in future runs without changing the optimization pipeline.

3.8 Training Data

GLiNER-Relex is trained through large-scale synthetic annotation followed by smaller high-quality fine-tuning. The released checkpoint uses a DeBERTa-v3-large encoder, all-pairs entity construction, and documented inference thresholds.

  • Training Data: Training data are constructed in two stages from FineWeb text using LLM-based annotation.The first stage provides large-scale synthetic data, while the second adds a smaller curated set of higher-quality examples.
  • Stage 1: Large-scale pre-training data: Stage 1 annotates approximately 1 million sentences and 50,000 full-length texts with entities and relation triplets.Sentence-level and document-level examples are mixed into one training set to include document context.
  • Stage 2: High-quality fine-tuning data: Stage 2 curates approximately 3,000 high-quality examples through entity-rich passage filtering and two-pass Gemini annotation.The process first extracts coarse entity types, retains passages with sufficient entity density, and then performs further annotation.
  • Implementation: The released checkpoint uses a DeBERTa-v3-large shared encoder followed by a bidirectional LSTM with hidden size 1024.It supports sequences up to 2048 words and candidate spans up to 12 words.
  • Released checkpoint: The released model uses all-pairs enumeration, omits adjacency loss, and trains with λE = λR = 1.0, α = 0.75, and γ = 0.Inference uses entity and relation confidence thresholds of τE = 0.3 and τR = 0.5.

4 Experiments

GLiNER-Relex is evaluated in zero-shot relation extraction across four benchmarks against end-to-end systems and a specialized classifier supplied with gold entities. It achieves the strongest end-to-end results on DocRED and CrossRE, while performance is lower on FewRel.

  • Aggregate comparison: Across the four benchmarks, GLiNER-Relex averages 25.6% Micro-F1, versus 22.1% for GPT-5-mini and 17.1% for GLiNER2 among end-to-end systems.GLiREL’s scores fall to 4.5% on CoNLL04, 2.2% on DocRED, and 1.4% on CrossRE despite receiving gold entities.

5 Discussion

GLiNER-Relex unifies entity and relation extraction with zero-shot labels, shared representations, and a single forward pass. The discussion highlights efficiency and GraphRAG potential alongside limitations in dense and long-document settings.

  • GLiNER-Relex jointly extracts entities and relations in a single forward pass with explicit entity-pair modeling and zero-shot generalization.
  • The shared architecture eliminates error propagation between separate NER and RE stages.
  • Users can adapt extraction to new domains by specifying natural-language entity and relation labels without retraining.The inference API accepts separate entity and relation labels and confidence thresholds, returning entity spans and relation triplets.
  • GraphRAG is a promising application because encoder-based extraction could reduce time and cost budgets for large-corpus knowledge-graph construction.A systematic comparison with LLM-based GraphRAG extractors, including answer quality, graph coverage, and total cost, remains future work.
  • Zero-shot performance does not yet match fully supervised models or frontier LLMs on all benchmarks.The paper identifies fine-grained relation inventories as a setting where hierarchical or prototype-based representations may improve performance.
  • Precision degrades in entity-dense passages because all-pairs enumeration creates quadratic candidate pairs and more spurious predictions.Adjacency-guided selection mitigates this issue, but dense entity graphs remain challenging in biomedicine, legal text, and financial reports.
  • Long-document extraction is constrained by short pretraining sequences, quadratic pair growth, and pressure on the fixed-dimensional embedding space.Suggested directions include hierarchical encoding, cross-chunk attention, bi-encoder extensions, entity linking, and adjacency-decoder ablations.

6 Conclusion

GLiNER-Relex provides a unified, efficient framework for zero-shot joint entity and relation extraction. Its natural-language label interface supports structured knowledge extraction across domains and offers a promising backbone for knowledge graphs powering RAG systems.

  • GLiNER-Relex extends GLiNER with a dedicated relation extraction module for joint named entity recognition and relation extraction.
  • The model achieves competitive zero-shot performance on standard relation extraction benchmarks while maintaining encoder-based computational efficiency.
  • Users can specify arbitrary entity and relation types through natural-language labels, enabling extraction across diverse domains without retraining.
  • GLiNER-Relex is presented as an efficient solution for extracting structured knowledge from unstructured text for knowledge graphs, document understanding, and information extraction.
Loading 2605.10108v1…