Source-linked AI summary
GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer
Urchade Zaratiana, Nadi Tomeh, Pierre Holat, Thierry Charnois
TL;DR
NER systems must balance broad entity-type coverage with practical computational demands. GLiNER uses a compact bidirectional model that matches entity types to text spans, and it reports stronger zero-shot performance than ChatGPT and fine-tuned LLMs across several evaluations. Its multilingual results include languages absent from training, although performance is weaker in some non-Latin languages.
Problem
Traditional NER models are restricted to predefined entity types, while flexible LLM-based extraction can require large computational resources and sequential decoding.
Method
GLiNER uses a bidirectional language model to match entity-type embeddings with textual-span representations, trained on diverse Pile-NER data.
Results
GLiNER outperforms ChatGPT and fine-tuned LLMs in zero-shot NER evaluations, including unseen languages.
Takeaways & Limitations
GLiNER provides a resource-efficient alternative for open-type NER across diverse domains and languages.
Takeaways & Limitations
Performance is less competitive in non-Latin languages, reaching an F1 score of 0.89 in Bengali.
Abstract
from arXiv · showhide
Named Entity Recognition (NER) is essential in various Natural Language Processing (NLP) applications. Traditional NER models are effective but limited to a set of predefined entity types. In contrast, Large Language Models (LLMs) can extract arbitrary entities through natural language instructions, offering greater flexibility. However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity. Leveraging a bidirectional transformer encoder, our model, GLiNER, facilitates parallel entity extraction, an advantage over the slow sequential token generation of LLMs. Through comprehensive testing, GLiNER demonstrate strong performance, outperforming both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks.
1 Introduction
Open-type NER offers flexibility beyond predefined entity types, but LLM-based approaches can be costly and slow. GLiNER addresses these constraints with a smaller bidirectional model that matches entity types to text spans and performs strongly in zero-shot evaluation.
- Motivation: Traditional NER models use predefined entity types, whereas LLMs can identify arbitrary entity types from natural-language instructions.This flexibility is useful for applications such as knowledge-graph construction, but expanding traditional labels requires additional dataset annotation.
- Approach: GLiNER prompts a bidirectional language model for open NER instead of prompting an LLM through multi-turn dialogue.The model uses learned representations to match entity types with text spans.
- Motivation: Autoregressive LLM approaches require token-by-token decoding, large models, and cannot predict multiple entity types in parallel.These properties limit deployment in compute-constrained settings and make extraction slower.
- Approach: GLiNER uses a smaller bidirectional language model and formulates open NER as matching entity-type embeddings with textual-span representations in latent space.This reframes extraction as matching rather than generation and supports bidirectional context processing.
- Results: GLiNER outperforms ChatGPT and fine-tuned LLMs on zero-shot NER datasets and surpasses ChatGPT in 8 of 10 unseen languages.The model was trained on data spanning numerous domains and thousands of entity types.
2 Method
GLiNER performs open-type NER by matching natural-language entity types with textual span representations in a shared latent space. Its bidirectional encoder, parallel span computation, and task-specific decoding support efficient extraction.
- Architecture: GLiNER uses a bidirectional language model, span representation module, and entity representation module to extract arbitrary entity types.Entity and span embeddings are placed in the same latent space so their compatibility can be assessed.
- Input format: The input combines natural-language entity types and source text, separated by a [SEP] token and prefixed with [ENT] tokens.The [ENT] tokens are placed before entity types and initialized randomly at training start.
- Token representation: The encoder produces contextualized representations for entity types and input words, using the first subword representation for words split into multiple subwords.This unified encoding lets entity-type and text representations interact bidirectionally.
- Span representation: Span embeddings are computed with a two-layer feedforward network over concatenated token representations, with span computation parallelized.A maximum span length of K=12 preserves linear complexity without harming recall.
- Matching and training: Matching scores estimate whether each span corresponds to an entity type, and sigmoid outputs are interpreted as probabilities under binary cross-entropy training.Training increases scores for positive span-type pairs and decreases them for negative pairs.
- Decoding: Greedy decoding selects spans with matching scores above 0.5 while enforcing flat or nested NER constraints and using a priority queue with O(n log n) complexity.Flat NER avoids overlap; nested NER permits fully nested spans while avoiding partial overlap.
3 Experimental Setting
GLiNER is trained on diverse automatically constructed data and evaluated zero-shot across out-of-domain, multi-domain, and multilingual NER benchmarks. The experiments compare it with prompted and fine-tuned open-type NER models using exact-match F1-score.
- Training data: Pile-NER supplies diverse training data by using ChatGPT to extract unspecified entity types from 50,000 texts sampled from the Pile corpus.After filtering, the dataset contains 44,889 passages, 240k entity spans, and 13k distinct entity types.
- Implementation: GLiNER uses deBERTa-v3 as its backbone, with separate learning rates for pretrained and non-pretrained layers and training for up to 30k steps.The setup uses AdamW, 10% warmup, decay, and dropout rate 0.4.
- Baselines: Evaluation compares GLiNER with ChatGPT, Vicuna, InstructUIE, UniNER, GoLLIE, and other open-type NER systems.The baselines include both prompted chat models and fine-tuned large language models.
- Benchmarks: Zero-shot evaluation covers seven out-of-domain datasets, 20 datasets spanning domains such as biomedicine and news, and multilingual data in 11 languages.The benchmarks assess out-of-domain generalization, broad domain coverage, and multilingual performance.
- Metric: Performance is measured with F1-score using exact matches between predicted and actual entities.This is the standard NER evaluation methodology adopted in the experiments.
4 Results
GLiNER is evaluated zero-shot on OOD and diverse NER benchmarks without target-dataset fine-tuning. Its compact models perform strongly against substantially larger general-purpose and NER-specialized baselines.
- Zero-shot evaluation trains GLiNER only on Pile-NER, without further fine-tuning on target datasets.
- OOD NER Benchmark: GLiNER is evaluated on an OOD benchmark using small, medium, and large model variants against multiple baselines.
- OOD NER Benchmark: 90M GLiNER-M matches UniNER-13B at 55 F1 while using 140 times fewer parameters, and GLiNER-L exceeds UniNER-13B by an average of 5 points.
20 NER Benchmark
Across 20 diverse NER datasets and multilingual zero-shot evaluation, GLiNER is broadly competitive and often superior, while performance varies by domain, language, and training setup. Its strongest results include leading 13 datasets zero-shot and outperforming InstructUIE after supervised fine-tuning, but it trails UniNER in some settings.
- Zero-shot English datasets: GLiNER achieves the highest performance on 13 of 20 datasets and surpasses UniNER by an average of 2 points.
- Zero-shot English datasets: GLiNER underperforms UniNER on tweet-based NER datasets, indicating weaker performance on informal or noisy social-media text.
- 20 NER Benchmark: The 20-dataset benchmark spans biomedical, news, and tweet domains, while multilingual evaluation uses the Multiconel dataset.
- Zero-Shot Multilingual Evaluation: GLiNER-Multi surpasses ChatGPT in most evaluated languages despite training on the English-only Pile-NER dataset.
- Zero-Shot Multilingual Evaluation: GLiNER-En is less competitive in non-Latin languages, scoring 0.89 F1 on Bengali.
- In-domain Supervised Finetuning: In-domain pretrained GLiNER exceeds the non-pretrained variant by 0.8 and InstructUIE by 0.9 on average, but trails UniNER by almost 3 points.
5 Further analysis and ablations
The ablations examine backbone choice, pretraining, negative entity sampling, and entity type dropping. deBERTa-v3 performs best across evaluated benchmarks, while Pile-NER pretraining and type dropping improve performance, especially with limited supervision.
- 5.1 Effect of Different Backbones: deBERTa-v3 achieves the highest performance on both the OOD and 20 NER benchmarks, outperforming the other tested pretrained BiLMs.ELECTRA and AlBERT perform slightly lower, while BERT and RoBERTa have similar lower scores.
- 5.1 Effect of Different Backbones: Even BERT-base reaches around 49 F1 on the OOD benchmark, 2 F1 points above the average for ChatGPT and InstructUIE.
- 5.2 Effect of Pretraining on In-domain Performance: Pile-NER-pretrained models consistently outperform models trained only on supervised data, indicating positive transfer across the 20 NER datasets.
- 5.2 Effect of Pretraining on In-domain Performance: 5.6 performance points separate pretrained and supervised-only models when using 100 samples per dataset, with the gap shrinking as dataset size increases.
- 5.3 Ablations: Negative entity sampling evaluates training conditions with 0%, 50%, and 75% negative entities to better reflect passages where some entity types are absent.
- 5.3 Ablations: Randomly varying the number of entity prompts during training improves out-of-domain evaluation by an average of over 1.4 points.
6 Related Works
Related work traces NER from rule-based and sequence-labeling systems through span classification, question answering, and generation. Recent zero-shot approaches use prompted or fine-tuned large autoregressive models.
- Named Entity Recognition: Early NER systems used handcrafted rules and gazetteers, but had limited scalability and adaptability to new domains or languages.
- Named Entity Recognition: NER later shifted across sequence labeling, span classification, question answering, and generation formulations.
- Zero-shot learning for NER: Large autoregressive models have enabled zero-shot NER through natural language prompting.
- Zero-shot learning for NER: Other work fine-tunes large language models to better align their capabilities with entity recognition and information extraction.
7 Conclusion
The conclusion presents GLiNER as a bidirectional-language-model method for identifying varied entity types. It reports strong zero-shot performance, resource efficiency, and multilingual capability, including languages absent from training.
- Conclusion: GLiNER identifies varied entity types using bidirectional language models and outperforms ChatGPT in zero-shot scenarios.
- Conclusion: GLiNER offers a more resource-efficient alternative for environments with limited computing power.
- Conclusion: GLiNER performs well in multiple languages, including languages it was not trained on.
- Conclusion: Future work aims to improve GLiNER’s design and adapt it further for low-resource languages.