Source-linked AI summary
Unapologetically Distributed: A Call for Decentralized Document Analysis
Adrià Molina, Oriol Ramos Terrades, Josep Lladós
TL;DR
Centralized document-data aggregation can be impractical amid privacy constraints, while distributed learning has raised concerns about robustness to heterogeneous data. This study evaluates distributed learning across tasks, architectures, and fine-tuning strategies, reporting advantages particularly in out-of-domain and limited-parameter settings.
Problem
Centralized training can be impractical for sensitive document data, and distributed learning has faced concerns about performance under non-identically distributed data.
Source: §1 · pp. 1–2Method
The paper compares distributed and centralized learning across Document Analysis tasks, architectures, and fine-tuning strategies, including post-hoc parameter-level model merging.
Source: §1 · pp. 2–3Results
Across tasks, distributed learning performs best with fewer learnable parameters; it consistently dominates in out-of-domain Word Spotting, while Table Recognition is strongest zero-shot.
Source: §6 · p. 14Takeaways & Limitations
The authors position distributed learning as a targeted option for low-resource, limited-adaptability, or constrained deployments, with historical Document Analysis particularly aligned to this paradigm.
Source: §6 · p. 14The stated personalization hypothesis compares distributed pre-training with centralized training under identical hyperparameter settings and epoch counts.
Source: §3.2 · p. 6
Abstract
from arXiv · showhide
Privacy has become an increasingly important concern in the Document Analysis community, to the extent that in many environments such as archives, governmental institutions, and local businesses, the adoption of automation is restricted by legal and policy constraints. While federated learning has often been regarded as a ``necessary evil'', implying an unavoidable performance trade-off in exchange for decentralization and privacy, many prior works overlook its potential to improve robustness to out-of-distribution data. In this paper, we present Unapologetically Distributed, the first comprehensive study evaluating distributed learning in Document Analysis along three key axes simultaneously: the tasks addressed, the architectures employed, and the fine-tuning strategies applied. Specifically, we demonstrate how various distributed training approaches enhance generalization capabilities across diverse tasks such as Table Recognition, handwriting recognition, and Word Spotting, particularly during transfer learning stages. Our results provide strong evidence that decentralization is not merely a constraint, but a valuable opportunity to improve model robustness and adaptability in real-world Document Analysis scenarios.
1 Introduction
Centralized document-data aggregation can be impractical where privacy, resource, or preservation constraints apply. The paper evaluates distributed learning across tasks, architectures, and fine-tuning strategies to identify when it offers performance benefits.
1 Introduction
- Distributed learning enables collaborative model training without direct data sharing, while post-hoc parameter-level model merging is the approach used here.The paper positions this as an alternative for privacy-constrained and low-resource environments, where conventional centralized training may be impractical.Source: §1 · pp. 1–2
- The study targets conditions where decentralized learning may be most effective, particularly low-resource, distribution-shifted, and operationally constrained scenarios.The authors state that computational budgets are controlled to isolate the effect of decentralized training and characterize its advantages.Source: §1 · p. 3
- Its evaluations include Graph Neural Network Table Recognition, cross-modal distillation for Word Spotting, and personalized layer-wise fine-tuning for multi-script handwriting recognition.The Table Recognition work includes end-to-end, fine-tuning, and zero-shot evaluations; the listed tasks address distinct training scenarios.Source: §1 · p. 3
- The study evaluates distributed learning across Document Analysis tasks, neural architectures, and fine-tuning strategies to characterize when it benefits performance over centralized approaches.The authors describe this as a comprehensive evaluation and state that their aim is to identify effective regimes, not advocate decentralization universally.Source: §1 · pp. 2–3
2 Related Work
Prior work often frames distributed learning as approximating centralized performance, while this paper treats it as a paradigm to evaluate in its own right. The related-work discussion also identifies Table Recognition as an unaddressed federated-learning task and distinguishes this study's focus from related aggregation approaches.
2 Related Work
- Prior distributed document-learning studies often frame the approach as matching centralized performance, rather than as a first-class learning paradigm across domains.The discussion cites FedOCR as an example of this framing and notes similar conclusions in work applying distributed learning to Graph Neural Networks [12].Source: §2 · p. 3
- To the authors' knowledge, no prior study had addressed federated learning for Table Recognition, despite the sensitivity of such data in industrial settings.Source: §2 · p. 4
- The paper presents its contribution as a holistic evaluation across tasks and architectures, including Table Recognition and knowledge distillation from federatedly learned systems.It distinguishes its approach from FedAvg through the absence of multiple federated rounds, and from MAML through domain-specific dataset splits.Source: §2 · p. 4
3 Methodology
The methodology evaluates distributed learning across three fine-tuning strategies, using independently trained and merged models to assess their performance in different Document Analysis scenarios.
3.1 Model Merging Strategy
- A distributed model is formed by independently training same-architecture models on data subsets and averaging their parameters while keeping the data local.Source: §3.1 · pp. 4–5
3.2 Finetuning Strategy
- The study tests distributed models through frozen-encoder distillation, layer-wise HTR/OCR fine-tuning, and end-to-end graph-based Table Recognition.Source: §3.2 · p. 5
- For Word Spotting, merged pretrained OCR encoders act as frozen visual teachers for text encoders trained to align embeddings.The text encoders use a <RET> token and are trained by minimizing embedding distance with Triplet Margin Loss.Source: §3.2 · p. 5
- For unseen alphabets, Latin-pretrained models are fine-tuned with limited data by updating later layers and replacing the language head with a dictionary-sized output layer.Source: §3.2 · pp. 5–6
- For graph-based Table Recognition, independently trained GNN message-passing layers are merged, then fine-tuned and evaluated on an unseen dataset.Transfer requires identical node representations and edge construction across source and target tasks.Source: §3.2 · p. 7
4 Experimental Setup
The setup defines pretraining and dataset-specific fine-tuning, then compares centralized training with independently trained models merged at the parameter level. It evaluates diverse document tasks and datasets using task-specific metrics.
4.1 Datasets
- Word-spotting teachers are trained on centered Latin-alphabet text, then distilled into text encoders and fine-tuned on in-domain, out-of-domain, and unseen-alphabet datasets.The study includes printed artifacts, variable fonts, visual occlusions, non-Latin scripts, and cipher datasets among the out-of-domain targets.Source: §4.1 · p. 8
- The study covers 27 evaluation datasets, including table-recognition benchmarks and M962, a private dataset with only 50 page-level annotations.Table-recognition datasets are simplified to binary table/not-table segmentation.Source: §4.1 · pp. 7, 10, Table 1 · p. 9
- Multi-script recognition starts from Latin-trained OCR models and personalizes them on Arabic, Chinese, Japanese, Korean, Hindi, Bangla, and cipher datasets.The evaluation includes both language-specific systems and multilingual models trained jointly on new alphabets.Source: §4.1 · p. 9
4.2 Metrics and evaluation
- Centralized models train on all available data, while distributed models merge individually trained models; the protocol matches total forward/backward steps and fine-tunes each model for 25 epochs.The fine-tuned models are tested on the corresponding dataset's test partition.Source: §4.2 · p. 10
- Evaluation uses node and edge accuracy for table recognition, mAP for Query-by-String retrieval, and word-level accuracy for text recognition.Query-by-String retrieval ranks word images against text queries; word-level accuracy requires a perfect transcription.Source: §4.2 · pp. 10–11
5 Results
Across word spotting, text recognition, and table detection, distributed learning performs best in several out-of-domain or low-resource settings, while its advantages vary with task and adaptation. In word spotting, out-of-domain fine-tuning consistently favors distributed pretraining; in table detection, the advantage shifts toward centralized training as data and adaptation increase.
5.1 Word Spotting
- 5% average improvement in mAP is reported for word spotting; out-of-domain fine-tuning consistently favors distributed pretraining, while in-domain results are mixed.Six of eight Latin datasets benefit in at least one architecture, and gains are larger when fine-tuning alphabets match pretraining.Source: §5.1 · p. 12
5.2 Optical Text Recognition
- Distributed text-recognition personalization yields an average best-case improvement of ×1.62 over centralized and baseline approaches; centralized models recover Borg accuracy in multilingual training.The performance gap narrows with multilingual training, and distributed models still outperform centralized models in those settings.Source: §5.2 · p. 12, Table 3 · p. 13, Table 4 · p. 13
5.3 Table Detection
- In table detection, distributed models lead in zero-shot node and edge accuracy, results are mixed with partial fine-tuning, and end-to-end fine-tuning generally favors centralized models except on M96.The distributed advantage is strongest with no adaptation or limited data; as data availability and adaptation capacity increase, results shift toward centralized training.Source: §5.3 · pp. 12–13
6 Conclusions
The conclusions identify low-resource and limited-data settings as especially favorable for distributed pre-training, while emphasizing that its benefits depend on task, data regime, and fine-tuning strategy. Historical document analysis, archives, and SMEs are highlighted as relevant settings under annotation, privacy, or access constraints.
6 Conclusions
- In Word Spotting, distributed approaches consistently dominate out-of-domain, while in-domain fine-tuning results are mixed.GRU- and LSTM-based text encoders show especially pronounced benefits relative to Transformer-based architectures.Source: §6 · p. 14
- Independent language training yields larger character-recognition advantages than joint multilingual training, where greater data availability narrows the performance gap.The conclusions highlight historical document analysis, archives, and SMEs as relevant applications under annotation, privacy, or access constraints.Source: §6 · p. 14
- Distributed pre-training is especially advantageous with limited data, particularly when architectures leave little room for fine-tuning or data are distribution-shifted.Table 6 summarizes the performance trends; the discussion identifies parameter-efficient and low-resource regimes as especially favorable.Source: §6 · p. 14, Table 6 · p. 14
- Table Recognition is strongest in zero-shot settings, mixed with partial fine-tuning, and degrades under full end-to-end fine-tuning.The largest gains occur on the smallest dataset, M96.Source: Table 5 · p. 13, §6 · p. 14
Supplementary Material
The supplement responds to review questions about the study’s merging protocol, statistical evidence, claim strength, and possible explanation for generalization. It reports bounded benefits rather than universal superiority and presents parameter averaging’s mechanism as an intuition, not a full theory.
Supplementary Material
- The supplementary material presents responses to questions raised during review and notes that some may be useful to readers.Source: Supplementary Material · p. 18
Single-round merging as distributed learning
- Single-round merging links FedAvg, task arithmetic, model soups, and 1-epoch meta-learning; rounds 2–5 shifted node/edge accuracy by at most 1–2 points without changing the winning regime.FedProx and SCAFFOLD add no distinct comparison under the one-round protocol because they collapse to FedAvg there.Source: Single-round merging as distributed… · p. 18, Table 7 · p. 19
Statistical significance
- Across 22 datasets, Table 2 reports paired centralized/distributed Word Spotting comparisons across many tasks, and a one-sided Wilcoxon signed-rank test over all paired runs confirms significance.Table 7 shows small spreads across five independent aggregation repetitions.Source: Statistical significance · p. 18
×1.62 and claim strength
- Negative in-domain gains mean distributed learning is not claimed to be universally superior; ×1.62 is labeled best-case, while Dist.(random) starts from scratch and reaches ×1.04.All Table 3 rows, including Centr.(base), share HierText pretraining; only Dist.(random) starts from scratch.Source: ×1.62 and claim strength · p. 18
Why does merging generalize better?
- Parameter averaging is offered as an intuition: dataset-specific directions may cancel while shared task-relevant directions survive, but a full theoretical treatment is outside scope.The authors connect this intuition to stronger benefits when downstream tasks or architectures leave little room to re-specialize.Source: Why does merging generalize better? · p. 18