Deep Learning Approaches on Image Captioning: A Review

• Image captioning combines natural language processing and computer vision to generate sentences describing an input image’s visual contents automatically.The field has applications including human-computer interaction, medical captioning, traffic analysis, industrial quality control, and assistive technologies.Source: §1 · p. 1
• Recent advances motivate an updated review; Hossain et al.’s comprehensive survey inspired this paper’s structure.Source: §1 · p. 1
• The review focuses on whole-image captioning, organizing methods by category rather than pairwise comparison and addressing vision-language pre-training that earlier surveys seldom explored.It responds to gaps in prior surveys, which often covered work only through 2018, included limited research, or gave little methodological detail.Source: §1 · p. 2
• The paper reviews image-captioning methods published from 2018 to 2022, analyzing datasets, evaluation metrics, method performance, challenges, and future directions.Its challenges section surveys inherent difficulties and potential solutions.Source: §1 · p. 2

2.1 Most Common Solutions

• Encoder-decoder methods map an image to a representation and then a caption; CNN encoders and RNN decoders are common, but compressing image information into one vector can yield vague captions.Attention- and graph-based methods appear more often than dense captioning in works from 2018 to 2022.Source: §2.1 · p. 2

2.2.1 R-CNNs

• R-CNN uses selective search for region proposals, Fast R-CNN convolves the image once, and Faster R-CNN learns proposals for real-time detection.Fixed-size grids are inadequate for objects of varied shapes and locations; selective search may generate incorrect proposals, while Faster R-CNN avoids it with a separate proposal network.Source: §2.2.1 · p. 3

2.2.2 RNNs

• RNNs use internal state to process dependent sequences, but can suffer vanishing gradients, difficult training, and information loss on long sequences.Source: §2.2.2 · p. 3

2.2.3 LSTMs and GRUs

• LSTMs use gates to preserve long-term dependencies and are common caption decoders, while GRUs [20] use fewer gates and can train faster.LSTMs ignore sentence hierarchy and require substantial storage; GRUs have performed better on some smaller, less frequent datasets.Source: §2.2.3 · p. 3

2.2.4 ResNet

• ResNets use stacked residual blocks and skip connections to make optimization easier and faster, and have been used to detect objects and generate image representations.Source: §2.2.4 · pp. 3–4

3 Deep Learning-Based Image Captioning

• The survey organizes and classifies image-captioning frameworks by their core structure and presents the taxonomy in Figure 1.It also adjusts some terminology and notation for consistency across the reviewed papers.Source: §3 · p. 4, Figure 1 · p. 4

3.1 Attention-Based Methods

• Attention-based captioning emphasizes relevant image regions at each decoding step, using weighted image features to guide the decoder [10].A context vector combines encoder hidden states according to attention weights; most methods use a CNN or region-based CNN encoder and an RNN decoder.Source: §3.1 · pp. 4–5
• Multi-head attention runs attention in parallel, concatenates and transforms its outputs, and can capture sequence parts with different dependency lengths.The passage describes queries, keys, and values as packed into matrices but does not provide a complete definition in the supplied text.Source: §3.1 · p. 6
• Soft attention weights image features and uses regular backpropagation, whereas hard attention samples stochastically and estimates gradients by averaging results with Monte Carlo.Hard-attention accuracy depends on sample count and quality; soft attention assumes a weighted average represents the attended area well.Source: §3.1 · p. 6
• The encoder-decoder approach uses a CNN image encoder and RNN sentence decoder, while Anderson et al.’s bottom-up/top-down method combines region proposals with two LSTMs [7].The bottom-up module uses Faster R-CNN to propose salient regions; the top-down visual-attention LSTM weights them, and a language-model LSTM generates the caption.Source: §3.1 · pp. 6–7
• Further attention extensions include Gu et al.’s coarse-to-fine decoders, AoA’s gated filtering of attention results, and RFNet’s multi-CNN recurrent fusion.The coarse-to-fine model passes attention weights and hidden vectors across three LSTMs and uses stacked attention to extract finer details; AoA gates information in both encoder and decoder, while RFNet applies multi-attention to fused thought vectors.Source: §3.1 · pp. 7–8

3.1.1 Injecting Spatial and Semantic Relation Information into Attention-Based Methods

• Spatial- and semantic-relation methods enrich attention: X-Linear blocks model second-order region–query interactions, while Cornia et al. condition region attention on ordered sentence chunks [23].X-Linear uses bilinear pooling and spatial and channel-wise attention; the chunk-based model uses syntactic dependencies and a gate to shift between region sets during generation.Source: §3.1.1 · pp. 8–9
• Adding spatial and semantic relations improves caption quality, but ambiguity, weak grounding, object-detector computation, and bounding-box annotation requirements remain [105, 23].Classic attention also fails to model relations between detected objects, motivating relation-aware approaches.Source: §3.1 · p. 8, §3.1.1 · p. 9

3.2 Graph-Based Methods for Spatial and Semantic Relations between Image Elements

• Scene-graph captioning uses image- or text-derived graphs to represent element relations; symbolic scene abstractions can make language generation less dependent on visual perception [25] [6].Semantic information and object attributes have also been shown to support higher-quality captions [36].Source: §3.2 · p. 9
• Yang et al. introduce SGAE, which connects image and text through scene graphs and sentence self-reconstruction to learn a shared dictionary encoding language inductive bias; a multimodal GCN complements missed visual cues.The intermediate representation D acts as working memory to re-key encoded image nodes into a more generic representation with smaller domain gaps.Source: §3.2 · p. 9
• Gu et al. train captioning without image-caption pairs by mapping image and sentence scene graphs into aligned feature spaces with Cycle-Consistent Adversarial Training, unlike the paired framework in.The framework uses image- and sentence-scene-graph generators and CycleGAN for unsupervised cross-modal feature alignment.Source: §3.2 · p. 9
• Gao et al. divide captioning into concept cognition and sentence reconstruction, using a CNN-RNN-SVM to generate a semantic scene-graph sequence that an RNN converts into captions.The CNN extracts visual features, the RNN models image/concept relationships and concept dependencies, and the SVM selects relevant semantic concepts.Source: §3.2 · p. 10
• The general workflow extracts image features, builds a spatial-semantic graph over detected regions, encodes regions and relations with GCNs, and passes the resulting vector to LSTM decoders.Graph vertices represent regions, while edges represent relationships between them.Source: §3.2 · p. 10, Figure 4 · p. 10

3.3 Combining Attention-Based Methods and Graph-Based Methods

• Yao et al. combine spatial and semantic graphs with graph convolution and an attention-based encoder-decoder to enrich image regions before LSTM caption generation.Faster R-CNN proposes regions; spatial and semantic relations form separate graphs, and decoder word distributions are linearly weighted at each step.Source: §3.3 · p. 10
• Zhong et al. decompose a scene graph into semantic sub-graphs, select meaningful components with an sGPN, and use an attention-based LSTM to generate sentences.The framework separates scene-graph extraction, sub-graph selection, and sentence decoding; generated tokens are associated with nodes in the selected sub-graph.Source: §3.3 · pp. 10–11
• Wang et al. combine graph-based relation representations with attention that remembers visited regions, while Chen et al. [19] use abstract scene graphs for user-controllable captions.Chen et al. gradually update graph representations to cover graph information without omission or repetition, using role-aware node embeddings and a multi-relational graph convolutional encoder.Source: §3.3 · p. 11
• Li et al. generate scene graphs and fuse region visual features with embedded relationship triplets through hierarchical attention during sentence generation.The visual features come from VGG-16 object regions, while relationship triplets provide semantic features.Source: §3.3 · p. 11
• Xu et al. combine scene-graph semantics, topology, and attended sub-graphs for an LSTM language model, while Lee et al. [75] attend directly to relation features without graph convolutions.Lee et al. present direct use of scene-graph relations as an alternative that avoids expensive graph convolutions.Source: §3.3 · pp. 11–12

3.4 Convolutional Network-Based Methods

• Convolutional methods extract image features with CNNs and use language-model output to generate captions, drawing on advances in convolutional image generation and machine translation.The cited sequence-to-sequence advances include convolutional image generation and machine translation [38].Source: §3.4 · p. 12
• Aneja et al. [8] replace recurrent LSTM or GRU units between input and output word embeddings with masked convolutions in a feed-forward model.The proposed approach contains three components: input word embeddings, masked convolutions, and output word embeddings.Source: §3.4 · p. 12
• Wang et al. [134] combine CNN-based vision and language modules with hierarchical attention to generate captions and process words in parallel.Their four modules are vision, language, attention, and prediction; the vision CNN uses VGG-16 without its fully connected layer, while the language CNN has no pooling. Attention is computed bottom-up at each language-model level and fed to the next; experiments found increasing kernel width a better way than adding layers to expand the receptive field.Source: §3.4 · p. 12
• Convolutional models can increase caption diversity and avoid vanishing gradients, but their evaluation-metric performance still needs improvement.The review also notes their stronger performance in classification tasks and that less attention has been paid to them than to previously discussed categories.Source: §3.4 · p. 12

Transformer-Based Methods

• Object Relation Transformer uses geometric attention to incorporate spatial relationships among detected objects into a Transformer encoder-decoder captioning model.Bounding-box properties are embedded with attention weights, while Faster R-CNN and ResNet-101 provide object detections and visual features.Source: §3.5 · p. 12
• GLIED globally models spatial and relational groupings of image regions and attribute words, then locally extracts fine-grained information to select words precisely.It uses region features from Faster R-CNN as provided by Anderson et al. [7].Source: §3.5 · p. 13
• M² [24] encodes low- and high-level image-region relationships with persistent memory and uses learned gating to combine them during non-recurrent caption generation.Its mesh-like encoder-decoder connections let the decoder attend to every encoding layer through scaled dot-product attention.Source: §3.5 · p. 13
• Transformer-based variants explore semantic and visual information through ETA-Transformer, raw-image encoding in CPTR [91], and concept-guided decoding in ViTCAP [32].PureT replaces Faster R-CNN with SwinTransformer and refines encoder-decoder features; COS-Net retrieves semantically similar sentences, filters and visually infers semantic cues, then orders them. COS-Net uses CLIP; ViTCAP predicts pseudo-ground-truth concepts with CTN and decodes using grid features and top-K concept tokens. COS-Net also introduces a semantic ranker, while SP and SR modules cluster grid features and combine low- and high-level features. Huang et al. pair a Transformer-like encoder with an LSTM decoder.Source: §3.5 · p. 13
• GRIT combines DETR-based detection with grid and region features, while ExpansionNet v2 [60] uses static expansion over sequences of different lengths for feature refinement.ExpansionNet v2 follows an encoder-decoder structure built on Swin-Transformer.Source: §3.5 · p. 13

• He et al. use Transformer layers with multiple sub-transformers to encode detected-region relations and decode information from visual-semantic and spatial graphs.Source: §3.6 · p. 13
• Chen et al. use visual and semantic sub-encoders with Faster-RCNN scene graphs, then feed a GCN-enriched semantic matrix into decoder multimodal attention.The attention module leverages multimodal representations for caption generation.Source: §3.6 · p. 13
• Yang et al. [146] propose ReFormer, a modified Transformer that generates relation-embedded features and explicitly represents pairwise object relationships while combining scene-graph generation with captioning.Source: §3.6 · p. 14

3.7 Vision Language Pre-Training Methods for Image Captioning

• Large-scale self-supervised pre-training aims to lessen reliance on fully supervised learning, then generalize the resulting model to downstream tasks.The pre-trained model learns from an enormous amount of data before downstream use.Source: §3.7 · p. 14
• CLIP [99] supplies shared image-text representations; ClipCap maps its embeddings to a caption prefix for GPT-2, while Barraco et al. use an encoder-decoder Transformer.ClipCap's learned prefix approach draws on Li et al.'s [82] proposal to adapt language models by concatenating a learned prefix.Source: §3.7 · p. 14
• VIVO pre-trains on image-tag pairs for object labeling and uses tags during caption fine-tuning, enabling zero-shot generalization to novel objects; XGPT is optimized directly for generation.XGPT uses a cross-modal encoder-decoder because vision-language pre-training methods for understanding cannot be directly applied to generation tasks.Source: §3.7 · p. 14
• Li et al. use salient object tags to anchor image-language alignment in a shared semantic space, while BERT-based methods [28] learn contextualized representations by predicting masked words or regions.The training examples pair a word sequence, object tags, and image-region features; the pre-training scheme models intra- and cross-modal relationships in image-text pairs.Source: §3.7 · p. 14
• Unified vision-language models include VLP for generation and understanding, mPLUG [76] with cross-modal skip connections, and Prismer [90] for domain-expert reasoning with less training data.VLP shares a Transformer for encoding and decoding and uses bidirectional and sequence-to-sequence masked-language prediction; mPLUG targets efficiency and information asymmetry, while Prismer performs well in fine-tuned and few-shot learning.Source: §3.7 · p. 14

3.8 Unsupervised Methods and Reinforcement Learning

• Early work used pivot-language translation, gradient-policy RNN reinforcement learning, and conditional GANs for diverse descriptions; Guo et al. added realism and style discriminators with sentence-level rewards.Guo et al. used an LSTM decoder as an agent that predicts successive words and updates its state after receiving a sentence-level reward.Source: §3.8 · p. 15
• Feng et al. [33] trained on independent images and a sentence corpus, using reconstruction in a shared latent space and a concept reward to encourage captions grounded in detected visual concepts.The model combines an image encoder, sentence generator, and discriminator; its sentence corpus uses Shutterstock captions.Source: §3.8 · p. 15
• CNN-based discriminators outperformed RNN-based discriminators, while RNN-based models saved 30% training time; ensembles of four models of each type improved on a single model [16].The framework uses a discriminator to distinguish human-described from machine-generated captions, addressing the limitation of optimizing a single metric.Source: §3.8 · p. 15
• Liu et al. used self-retrieval error to reward captions that match their input images better than distractor images, training the captioner through back-propagation and REINFORCE.A CNN extracts image features, an LSTM generates captions, and the retrieval module evaluates image-caption similarity against distractors.Source: §3.8 · p. 15
• Figure 5 depicts a general encoder-decoder workflow in which an LSTM generator receives rewards from an LSTM discriminator that distinguishes corpus sentences from generated captions.The workflow uses VGGNet as its image encoder.Source: §3.8 · p. 15, Figure 5 · p. 16
• Unsupervised methods aim to lessen dependence on paired image-caption datasets, and the survey notes that they are expected to become more favored given dataset-preparation and training difficulties.The section frames the field’s interest as a response to models’ dependence on the quality and volume of paired data.Source: §3.8 · pp. 15–16

3.9 Generating Multi-Style Captions

• Stylized captions can make image descriptions more engaging and support applications such as photo-sharing and chatbots.The cited styles include humorous, hostile, and poetic tones.Source: §3.9 · p. 16
• Shuster et al. introduced TransResNet, which projects images, captions, and personality traits into a shared space; their retrieval model outperformed their generative model.Retrieval selects a candidate caption from the dataset, while generation produces captions word by word.Source: §3.9 · p. 16
• Guo et al. combine image encoding, style-conditioned caption generation, adversarial caption discrimination, style discrimination, and back-translation from stylized to neutral captions.The back-translation module uses multilingual neural machine translation to translate stylized captions into neutral captions, encouraging content consistency.Source: §3.9 · p. 16
• Figure 6 presents ground-truth and generated caption examples from multiple categories, including Shuster et al. [121].Its examples also include captions by Huang et al. [65], Wang et al. [133], and Li et al. [81].Source: Figure 6 · p. 16, §3.9 · p. 16

4.1 The Exposure Bias Problem

• Exposure bias arises because models train on ground-truth previous words but must condition on their own predictions at test time, allowing errors to accumulate.Gu et al. used reinforcement learning with intermediate decoder outputs and previous-decoder outputs to normalize reward, addressing exposure bias and loss-evaluation mismatch.Source: §4.1 · p. 17

4.2 The Loss-Evaluation Mismatch Problem

• Loss-evaluation mismatch arises because models train with cross-entropy at each time step but are evaluated with non-differentiable sentence-level metrics.Reinforcement learning has been used to optimize these metrics.Source: §4.2 · p. 17

4.3 The Vanishing Gradient Problem

• Vanishing gradients can become so small that network weights do not change, potentially halting training completely.This problem occurs in neural networks trained with gradient methods and back-propagation.Source: §4.3 · p. 17

4.4 The Exploding Gradient Problem

• Exploding gradients can destabilize deep or recurrent networks with enormous weight updates, preventing learning and, in recurrent networks, learning over long sequences.In the worst case described, multilayer perceptrons can produce NaN weight values that can no longer be updated.Source: §4.4 · p. 17

4.5 Object Hallucination

• Object hallucination occurs when models describe objects absent from an image, harming users who prioritize correctness; CHAIR measures this problem at instance and sentence levels.Rohrbach et al. proposed CHAIRi and CHAIRs, respectively. Models optimized for CIDEr frequently hallucinated more, while GAN-based models reduced hallucination; attention models’ advantage was attributed mainly to access to convolutional features.Source: §4.5 · pp. 17–18

4.6 Illumination Conditions

• Poor or uneven illumination reduces contrast and obscures visual details, making objects and scenes harder to recognize and captions less accurate.Researchers explore contrast enhancement, color correction, and low-light image enhancement to mitigate these effects.Source: §4.6 · p. 18

4.7 Contextual Understanding

• Contextual understanding requires models to reason about object relationships, spatial arrangement, and the overall atmosphere of a scene.The section characterizes this as difficult because it requires deep understanding of visual content [36].Source: §4.7 · p. 18

4.8 Referring Expressions

• Referring expressions require models to identify and link objects, a difficult task when objects are obscured or similar, but they enable more detailed captions.Examples include descriptions such as “the girl with the red hair” or “the dog in the corner.”Source: §4.8 · p. 18

5.1 Using Attention

• Attention methods can select imprecise regions and single-stage models may produce limited captions, motivating adaptive proposals, coarse-to-fine decoding, AoA, and multiple visual encoders.Anderson et al. [7] propose bottom-up and top-down attention; Gu et al. pass earlier-stage attention weights and hidden vectors to later decoders; Huang et al. add a second attention level; and RFNet extracts multiple image views.Source: §5.1 · pp. 18–19

5.1.1 Injecting Spatial and Semantic Relation Information into Attention-Based Methods

• Attention-based systems can miss spatial and semantic relations, suffer ambiguity from overlapping detected regions and lack grounding, prompting relation-aware, anchored, unified, and controllable alternatives.Proposals include second-order interactions, GLIED, anchor-point pre-training, unified encoder-decoder VLP, and region ordering conditioned on syntactic structure [23].Source: §5.1.1 · pp. 19–20

5.2 Using Graphs for Spatial and Semantic Relations; 5.3 Using Attention and Graphs

• Graph-based methods encode image or text relations, but must address trivial object-list captions, multilingual data scarcity, grounding, and the cost of graph convolutions.Scene-graph approaches connect image and text modalities or align graphs and captions across languages; other methods select grounded subgraphs, retain previously attended regions, incorporate user intentions, or use visual relations directly instead of GCNs.Source: §5.3 · p. 20, §5.2 · p. 20

5.4 Using Convolutional Network-Based Methods

• Convolutional captioning models can train in parallel and avoid recurrent-network limitations, but their performance still needs improvement.Masked convolutions with attention outperformed an attention baseline, while another CNN-only framework was designed for parallel computation and faster training.Source: §5.4 · pp. 20–21

5.5 Using Transformers; 5.5.1 Using Graphs and Transformers

• Transformer methods aim for parallel, end-to-end captioning and detector-free representations, but graph-guided architectures and large-scale multimodal training remain computationally demanding.Approaches include full-Transformer encoders, grid representations, Deformable DETR with Swin, and pure Transformer designs; ReFormer and SGGC incorporate scene-graph relations, while more economical multimodal models are still needed.Source: §5.5 · p. 21, §5.5.1 · p. 21

5.6 Using Vision-Language Pre-Training for Image Captioning

• Vision-language pre-training is now a dominant approach that can improve zero-shot performance and reduce training time, while newer methods target generation, alignment, and computational costs.ClipCap uses CLIP embeddings as a prefix for GPT-2; VIVO learns alignment from image-tag pairs, and XGPT introduces generation-focused pre-training objectives. mPLUG and Prismer address inefficiency, information asymmetry, and data insensitivity with cross-modal fusion and lightweight expert components.Source: §5.6 · p. 22

5.7 Using Unsupervised Methods and Reinforcement Learning

• Unsupervised captioning addresses limited language coverage and dataset scale, while reinforcement learning and discriminators offer alternatives to metric-only optimization.Early unsupervised work improved caption diversity but sacrificed overall performance, and one proposed model was criticized as unsatisfactory; a self-retrieval module using REINFORCE improves a captioning system trained on partially labeled data.Source: §5.7 · pp. 22–23

5.8 Captioning in Multiple Styles

• Stylized captions can enrich engagement, but factual neutral descriptions may be more appropriate for visually impaired users than humorous or hostile styles.Retrieval-based systems may produce general, repetitive captions because they select from existing captions rather than generate new ones.Source: §5.8 · p. 23

6.1 Datasets

• MS COCO provides 328,000 images, 91 object classes, and five captions per image; Flickr30K Entities adds entity-linked captions and bounding boxes, while Visual Genome describes image regions and their relations.FlickrStyle10k adds poetic, humorous, and factual caption styles across its 10,000 images.Source: §6.1.1 · p. 23, §6.1.3 · p. 23, §6.1.4 · p. 23
• TextCaps focuses on text within images, Conceptual Captions supplies over 3.3 million filtered image-caption pairs, and Open Images adds visual relationships and word-aligned localized narratives.The nocaps benchmark [1] targets novel visual concepts, with about 400 test-image object classes having almost no associated training captions.Source: §6.1.6 · p. 24, §6.1.8 · p. 24, §6.1.9 · p. 24

6.2 Evaluation Metrics for Image Captioning Methods

• BLEU scores n-gram overlap and is fast and widely used, but can reward short outputs; ROUGE measures overlap with reference summaries, while METEOR combines weighted precision and recall with stemming and synonym matching.Text metrics assess generated text portions independently, unlike metrics designed specifically for image-caption evaluation.Source: §6.2.1 · pp. 25–26, §6.2 · p. 25, §6.2.2 · p. 26
• CIDEr uses TF-IDF-weighted n-gram consensus for image captions, while SPICE [6] scores semantic coverage of objects, attributes, and relations using scene graphs.The survey notes that these evaluation methods remain far from human judgments in quality, and that external knowledge databases may help.Source: §6.2.4 · p. 26, §6.2.5 · p. 26

6.3 Performance Comparison Based on MSCOCO Test Servers

• Li et al. achieves the best c5 and c40 results in most metrics on the MS COCO servers, while Aneja et al. [8] is lowest in almost all.Guo et al. has the worst BLEU-2 c5 and BLEU-3 c5 results; convolutional-network methods generally rank lower than attention-based methods in these comparisons.Source: §6.3 · pp. 26–27, Table 2 · p. 27, Table 3 · p. 27

6.4 Comparing Independent Results

• [157] leads independent results in BLEU-1 (90.7), BLEU-4 (59.3), METEOR (40.1), and ROUGE (71.5).These are the best results reported among the works covered by the survey.Source: Table 4 · p. 28, Table 5 · p. 28, Table 6 · p. 28
• [84] leads BLEU-2 (69.1) and BLEU-3 (54.9), while [23] leads CIDEr and SPICE (209.7 and 48.5, respectively).The survey identifies as COS-Net and [23] as Show, Control and Tell.Source: Table 4 · p. 28, Table 5 · p. 28, §6.4 · p. 28
• High-performing methods often use Transformers, scene graphs, and vision-language pre-training, which support long-range relations, structured visual relations, and joint visual-text learning.The survey associates these techniques with improved overall performance.Source: §6.4 · pp. 28–29

• Supervised captioners depend on dataset coverage, detector labels, and skewed image-caption examples, encouraging familiar captions over detected objects and language-prior-driven object hallucination.Source: §7 · p. 29
• Unpaired captioning faces image–sentence information misalignment because modality-specific encoders cannot be shared, and current unsupervised methods lag performance rankings.Source: §7 · p. 29
• Scene-graph captioning is hindered by difficult construction, non-pairwise interactions, tedious integration, and weak parsers; its benefit depends on pre-training generators with visually relevant relation data [75].Source: §7 · p. 29
• Vision-language pre-training (VLP) can address flaws in supervised and detector-based systems, but generation needs capabilities beyond understanding, while stronger detector-free captioners remain under-investigated [32].Source: §7 · p. 29
• Scene graphs, VLP, and Transformers feature in nearly all top-performing models, while graphs' ability to represent complex relationships may grow more important alongside LLMs and MLLMs [66] [5] [78].Source: §7 · pp. 29–30
• The survey sees unsupervised and unpaired learning as promising, expects graph methods to grow and LLMs, MLLMs, Transformers, and VLP to become standard, while noting little work on captioning for visually impaired users.Image descriptions could help visually impaired people understand their surroundings and identify potential dangers.Source: §7 · p. 30

8 Conclusion

• Captions remain below human quality, datasets cannot cover the real world, and current metrics do not precisely evaluate model performance.Despite many proposed methods, major challenges still have few suggested solutions.Source: §8 · p. 30
• Vision-language pre-training has shown promising performance and, with Transformers, is expected to become integral to future image-captioning models.Source: §8 · p. 30
• Visual assistants for visually impaired people remain underdeveloped; useful captions should prioritize important aspects, then describe surroundings and fine details such as texture and object positions.Captions may begin short and general, then become denser and more detailed in response to user questions.Source: §8 · p. 30