Domain Adaptation with Adversarial Training and Graph Embeddings

1 Introduction

• The motivating application is time-critical analysis of Twitter posts during sudden-onset natural or man-made disasters, where people report injuries, damage, and urgent needs.Source: §1 · p. 1
• The task is real-time classification of crisis tweets when labeled data for the new event is unavailable, using labeled and unlabeled data from past similar events and possibly unlabeled current-event data.Source: §1 · p. 1
• The unified framework combines a task-solving network with semi-supervised graph learning and adversarial domain adaptation to learn task-discriminative, domain-invariant representations.The graph encodes similarity between labeled and unlabeled instances, while a domain discriminator distinguishes source from target data.Source: §1 · pp. 1–2
• Adding the semi-supervised component gives 5% to 26% absolute gains in F1 over the supervised component, depending on the amount of labeled data.Source: §1 · p. 2
• Adversarial domain adaptation improves over the transfer-model adaptation baseline by 1.8% to 4.1% absolute F1.Source: §1 · p. 2
• Combining domain adversarial training with semi-supervised learning yields further gains of 5% to 7% absolute in F1 across events.The combined model builds on the gains reported for each component separately.Source: §1 · p. 2

2 The Model

• Shared convolutional and dense weights connect supervised, graph-based semi-supervised, and adversarial branches, whose losses support cross-domain tweet classification.The supervised loss uses labeled source data; the graph and adversarial losses use data from both events.Source: §2 · pp. 2–3

2.1 Supervised Component

• The supervised branch applies a task-specific dense layer to shared representations and uses labeled source tweets to train classification.Source: §2.1 · p. 3

2.2 Semi-supervised Component

• The semi-supervised branch uses a similarity graph to learn internal representations by predicting context nodes for input tweets.Source: §2.2 · p. 4, §2.2.1 · p. 4

2.2.1 Learning Graph Embeddings

• Negative sampling trains graph embeddings from two context types: graph-neighbor relations and shared labels among source instances.Positive and negative samples distinguish neighboring or same-label pairs from other pairs.Source: §2.2.1 · p. 4

2.2.2 Graph Construction

• With no relational knowledge source and pairwise distance computation considered expensive, the graph uses k-nearest neighbors over averaged word2vec vectors and Euclidean distance, with k=10.Source: §2.2.2 · p. 4

2.3 Domain Adversarial Component

• A domain discriminator distinguishes source from target tweets, while reversed discriminator gradients train shared features toward domain invariance.Source: §2.3 · pp. 4–5

2.4 Model Training

• Training alternates graph, source-supervised and adversarial, then cross-domain adversarial updates; fixed loss weights are λd=1e-8 and λg=1e-2.The pre-trained embedding matrix is kept fixed; tuning it took 3 days rather than 5 hours without significant performance gain. More sophisticated loss-weighting schemes are left for future work.Source: §2.4 · pp. 5–6

2.5 Crisis Word Embedding

• Pretrained CBOW embeddings use a crisis corpus of about 364 million tweets and yield about 2 million 300-dimensional word vectors.Training used a context window of 5 and 5 negative samples.Source: §2.5 · p. 6

3.1 Datasets

• The NEQ and QFL datasets comprise millions of event-keyword tweets, with 11,670 Nepal and 10,033 Queensland tweets sampled for labeling and the data split into training, test, and development sets.Workers labeled tweets relevant or non-relevant to crisis response; Table 1 reports class-wise distributions.Source: §3.1 · p. 6, Table 1 · p. 6

3.2 Model Settings and Baselines

• The ablation compares semi-supervised-only and domain-adaptation-only configurations with a combined model to examine the contributions of the network components.Source: §3.2 · p. 6

3.2.1 Settings for Semi-supervised Learning

• Semi-supervised experiments compare graph-based learning with self-training and supervised baselines, using 50K same-domain unlabeled instances and batches of 100, 500, 2,000, 5,000, or all labeled examples.Self-training uses predictions with confidence ≥0.75; supervised runs use the same number of labeled examples as the semi-supervised runs.Source: §3.2.1 · pp. 6–7

3.2.2 Settings for Domain Adaptation

• The transfer baseline trains a CNN on one event and tests on another; adaptation experiments use different source and target events, with only unlabeled target instances.Source: §3.2.2 · p. 7

3.2.3 Training Settings

• Training uses fixed CNN hyperparameters across settings, AdaDelta updates, loss-specific learning rates, dropout, and validation-based early stopping.The maximum is 200 epochs, dropout is 0.02, and early stopping uses F-measure with a patience of 25 epochs.Source: §3.2.3 · p. 7

3.2.4 Evaluation Metrics

• Performance is measured with weighted-average precision, recall, F-measure, and AUC to account for class imbalance.Source: §3.2.4 · p. 7

4.1 Semi-supervised Learning

• 4% to 13% absolute F1 improvements are reported for graph-based semi-supervised learning over supervised and self-training baselines on the Nepal and Queensland datasets.Source: §4.1 · p. 7
• F1 improves as labeled data increases, with 5% to 26% absolute gains over supervised models; gains are largest from 100 to 1,000 labels and comparatively minor afterward.The reported labeled-only F1 ranges from 43.63 to 60.89 for NEQ and from 48.97 to 80.16 for QFL; the authors describe the 500- and 1,000-label settings with unlabeled data as reasonable at event onset.Source: §4.1 · p. 8

4.2 Domain Adaptation

• 5% to 7% absolute F1 gains are reported when semi-supervised learning is combined with unsupervised domain adaptation.Table 4 presents domain-adaptation results across weighted-average AUC, precision, recall, and F-measure.Source: Table 4 · p. 8, §4.2 · p. 8
• Domain-adversarial training yields 1.8% to 4.1% absolute F1 gains across events, and the domain-adaptation experiments use only unlabeled target-domain instances.The authors foresee potential future improvement from using a small amount of labeled target-domain data.Source: §4.2 · p. 8

• Self-training labels unlabeled instances for retraining, while co-training uses two conditionally independent feature sets to train classifiers that iteratively label data for each other [Scudder, 1965; Mitchell, 1999].Source: §5 · p. 9
• Graph-based semi-supervised learning encodes instance similarity in edge weights; inductive methods learn a prediction function, unlike transductive methods, which predict only on observed instances.Source: §5 · p. 9
• The paper’s graph approach predicts contextual nodes using tweet similarity based on word embeddings, CNN features, and similarity-based selection rather than random walks [Yang et al., 2016].Source: §5 · p. 9
• Domain adaptation spans supervised, semi-supervised, and unsupervised methods, from linear kernels to deep networks, including source reweighting and noise-trained stacked denoising auto-encoders.Source: §5 · p. 9
• Domain adversarial neural networks learn discriminative, domain-invariant representations; this paper extends that approach by combining it with semi-supervised graph embedding for unsupervised domain adaptation.Source: §5 · p. 9
• Kipf and Welling (2016) applied graph convolution networks to citation and knowledge-graph datasets and reported better semi-supervised classification than other graph-based approaches.Source: §5 · p. 9

• Combining domain adversarial training with graph-based semi-supervised learning improves over the respective baselines on two crisis-related tweet datasets.The evaluation uses unlabeled target-event data and also examines batch-wise incremental graph training and approximate labeled-example needs at event onset.Source: §6 · p. 9