PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning-based synthetic-CT generation in radiotherapy and PET: a review

Maria Francesca Spadea, Matteo Maspero, Paolo Zaffino, Joao Seco

arXiv:2102.02734v2physics.med-phcs.LGeess.IV

TL;DR

This review examines deep learning-based synthetic-CT generation in clinical imaging contexts, including challenges related to online adaptive radiotherapy. It reports feasibility at several anatomical sites, dose differences below 1% versus CT-based planning, and better PET attenuation-correction performance than standard MRAC methods, while noting that limited FOV may hinder plan calculation.

  • Problem

    Daily CBCT has not commonly been used for online plan adaptation, and converting CBCT for this purpose presents a clinical challenge.

  • Method

    The review evaluates deep learning-based methods for synthetic-CT generation and considers multi-2D methods using slices from different views.

  • Results

    < 1% dose difference to CT-based planning is reported in the radiotherapy context, with better performance for PET attenuation correction than standard MRAC methods.

  • Takeaways & Limitations

    Synthetic CT seems feasible for sites such as head-and-neck/brain and pelvis, with potential utility in radiotherapy and PET attenuation correction.

  • Takeaways & Limitations

    Limited FOV may hinder calculating the plan for replanning purposes.

Abstract

from arXiv · show

Recently, deep learning (DL)-based methods for the generation of synthetic computed tomography (sCT) have received significant research attention as an alternative to classical ones. We present here a systematic review of these methods by grouping them into three categories, according to their clinical applications: I) To replace CT in magnetic resonance (MR)-based treatment planning. II) Facilitate cone-beam computed tomography (CBCT)-based image-guided adaptive radiotherapy. III) Derive attenuation maps for the correction of positron emission tomography (PET). Appropriate database searching was performed on journal articles published between January 2014 and December 2020. The DL methods' key characteristics were extracted from each eligible study, and a comprehensive comparison among network architectures and metrics was reported. A detailed review of each category was given, highlighting essential contributions, identifying specific challenges, and summarising the achievements. Lastly, the statistics of all the cited works from various aspects were analysed, revealing the popularity and future trends, and the potential of DL-based sCT generation. The current status of DL-based sCT generation was evaluated, assessing the clinical readiness of the presented methods.

I. Introduction

Radiotherapy and PET use complementary imaging modalities, but registration errors, CBCT artefacts, and missing attenuation information limit direct clinical use. This review systematically surveys deep learning-based synthetic CT generation across MR-only treatment planning, CBCT-guided adaptive radiotherapy, and PET attenuation correction.

  • CT provides high-resolution patient geometry and electron-density information for radiotherapy dose calculations, while MRI offers superior soft-tissue contrast.
  • MRI–CT registration and patient set-up differences can introduce systematic errors, motivating MR-only radiotherapy workflows.
  • MR-only radiotherapy is constrained by the lack of tissue attenuation information needed for accurate dose calculations, which synthetic CT aims to provide.
  • CBCT suffers from scatter noise, truncated projections, and artefacts, limiting routine online plan adaptation; CBCT-to-CT conversion may support dose computation and image-guided adaptive radiotherapy.
  • Synthetic CT can also address PET attenuation correction, including MRI-to-sCT translation for PET/MRI and derivation from uncorrected PET for standalone scanners.

I MR-only RT;

The review organises deep learning-based synthetic CT studies by input modality, network architecture, data pairing, volume-sampling configuration, and evaluation strategy. CNNs, U-Nets, GANs, and cycle-GANs support increasingly varied image-to-image translation and training setups.

  • II. MATERIAL AND METHODS: The review tables summarise anatomy, patient numbers, imaging protocols, architectures, volume-sampling configurations, paired or unpaired training, treatment, and evaluation metrics.
  • II.A. Deep learning for image synthesis: CNNs learn convolutional filter weights through training, with U-Nets and generative adversarial networks among the most popular architectures for image synthesis.
  • II.A. Deep learning for image synthesis: U-Nets combine encoding and decoding paths with skip connections, whereas GANs train a generator and discriminator using multiple objectives.
  • II.A. Deep learning for image synthesis: Cycle-GANs enable unpaired image-to-image translation by training forward and backward mappings with cycle-consistency losses.

II.B. Metrics

The review groups evaluation metrics into image similarity, geometric accuracy, and task-specific categories. These metrics compare synthetic CT with target CT or assess clinically relevant structures and outcomes.

  • Image similarity: Image-similarity metrics evaluate voxel-wise agreement between synthetic CT and aligned target CT, including MAE, RMSE, PSNR, SSIM, CC, and NCC.

II. MATERIAL AND METHODS II.B. Metrics

The review details how image-based, geometric, dose-based, proton-range, and PET metrics are computed. It also emphasises that metric choice, regions of interest, and analysis parameters complicate comparisons across studies.

  • Image similarity: MAE and RMSE average absolute or squared voxel differences, while PSNR and SSIM assess synthesis noise and structural similarity relative to target CT.
  • Geometric accuracy: Geometric evaluation derives segmented tissue or structure masks and compares them using DSC, Hausdorff distance, or mean absolute surface distance.
  • Metric interpretation: Even identical metrics can yield non-comparable results when regions of interest or implementation details differ, so metric selection should be application-specific.
  • Task-specific metrics: Radiotherapy task metrics include dose difference, dose pass rate, gamma pass rate, and dose-volume-histogram differences against CT-based calculations.
  • Task-specific metrics: Gamma-analysis comparisons are difficult because results depend on dose and distance criteria, analysis type, dose thresholds, grid size, and voxel resolution.
  • Task-specific metrics: Proton studies evaluate range-shift error using the 80% distal dose-fall-off depth, comparing prescribed and synthetic-CT-derived ranges.

III. Results

The review identified 83 eligible studies across three clinical applications, with MR-only radiotherapy comprising most publications. Across studies, 2D networks, paired training, regression objectives, and GAN architectures were common, while patient cohorts and evaluation practices varied.

  • 83 eligible papers were identified after duplicate removal and content screening.
  • 51 studies addressed MR-only radiotherapy, compared with 15 CBCT-to-CT studies and 17 PET attenuation-correction studies.
  • Publication numbers generally increased over time, except for CBCT-to-CT and PET attenuation correction, which were stable in recent years.
  • Most studies included fewer than 50 patients, although reported cohorts ranged from 10 patients or volunteers to much larger datasets.
  • Most models performed regression from the input image to sCT, while two studies used segmentation into predefined classes.
  • Paired training predominated, with unpaired training investigated in 13 of 83 articles; GANs were the most frequent architecture, followed by U-nets and other CNNs.
  • Across all categories, 2D networks were used about 61% of the time, followed by 3D configurations at 24%, 2.5D at 10%, and 2D+ at 6%.

III.A. MR-only radiotherapy

MR-only radiotherapy studies most often used 2D paired models and GAN architectures, while exploring MRI inputs, network dimensionality, and paired versus unpaired training. Reported dose agreement was generally high, but comparisons across studies were limited by inconsistent metric calculation.

  • MRI inputs: Multiple MRI sequences sometimes improved sCT generation, with contrast-enhanced T1 GRE producing the lowest MAE and highest PSNR in one brain study.
  • Model configurations: 2D models were the most common in MR-only radiotherapy, followed by 3D patch-based and 2.5D models.
  • Model configurations: 2D+ approaches were reported as beneficial relative to single 2D views, while studies comparing 2D and 3D models reached contradictory conclusions.
  • Training strategy: Paired training generally produced higher image similarity or lower dose differences than unpaired training, although mixed paired/unpaired approaches and structure-constrained losses were also beneficial in reported studies.
  • Architectures: GANs comprised approximately 55% of architectures, followed by U-nets at approximately 35% and other CNNs at approximately 10%.
  • Evaluation: Evaluation practice varied: 21 studies reported only image-similarity metrics, whereas 30 also assessed sCT-based dose calculation.
  • Evaluation: Overall, dose differences averaged below 1% and γ2%,2mm pass rates exceeded 95%, except in one study.

III.B. CBCT-to-CT generation

CBCT-to-CT generation is the most recent sCT application reviewed and targets image-guided adaptive radiotherapy. Studies evaluated image quality, segmentation, and dosimetric accuracy across several anatomical sites using predominantly paired or independently trained approaches.

  • CBCT-to-CT generation was proposed as an sCT application in 2016, with publication numbers rapidly increasing thereafter.
  • Paired training was used in eleven studies through rigid or deformable registration, whereas only three studies investigated unpaired training.
  • The reviewed workflow assessed image similarity, anatomical segmentation, and dosimetric accuracy, including photon and proton applications.
  • Most studies investigated head-and-neck and pelvic regions, while other work included thorax, breast, lung, and pancreatic sites.
  • A single network trained across head-and-neck, breast, and lung sites achieved image-similarity results similar to independently trained networks.

III. RESULTS III.B. CBCT-to-CT generation

CBCT-to-CT studies generally found that deep learning improved sCT quality and that model performance depended on architecture, training strategy, and anatomy. Evidence also suggested transferability across sites and scanners, but residual discrepancies remained for pancreatic radiotherapy.

  • Generalisation: A network tested on additional pelvic and head-and-neck cohorts achieved similar MAE across both testing groups, supporting transfer of pretrained models across anatomy.
  • Architectures: GAN configurations outperformed U-net in one comparison, while cycle-GAN exceeded DCGAN and PGGAN for image similarity in unsupervised training.
  • Architectures: Adding an attention gate to a 3D patch cycle-GAN improved performance over U-net and cycle-GAN without the gate for moving-organ pancreatic radiotherapy.
  • Evaluation: Deep learning significantly improved sCT relative to CBCT for the pancreatic site, although residual discrepancies remained.

III.C. PET attenuation correction

Deep learning methods generate PET attenuation-correction sCT from MR or uncorrected PET, with studies reporting improved PET accuracy and PETerr below 1% on average.

  • Fourteen studies investigated MR-to-CT image translation for PET attenuation correction, while three investigated uncorrected PET-to-CT.
  • Dixon, UTE and ZTE sequences are used to improve representation of short-T2 tissues such as cortical bone.
  • DL prediction reduced corrected-PET SUV RMSE by a factor of 4 for bone lesions and 1.5 for soft-tissue lesions.
  • Pre-training a U-net on 19 healthy brains enabled faster training with slightly lower residual error than random initialization.
  • DL methods outperformed atlas-based MRAC in DSC, MAE and PETerr, while CNN and cycle-GAN showed no significant difference.
  • For uncorrected-PET inputs, global MAE was about 110 HU versus 10–15 HU for site-specific MR-to-CT studies, but average PETerr remained below 1%.

IV. Discussion

The review finds broad promise for deep learning-based sCT across MR-only radiotherapy, CBCT-to-CT conversion and PET attenuation correction, while clinical implementation remains limited by validation and generalization requirements.

  • MR-only RT: Fast image translation, completed in a few seconds, supports the feasibility of applying DL-based sCT generation in clinical scenarios.
  • MR-only RT: MR-only radiotherapy was the largest category, with 51 papers demonstrating the potential of DL for MRI-based sCT generation.
  • MR-only RT: Dose differences below 1% were reported for photon and proton radiotherapy, including high-field MR-guided breast cases.
  • Clinical adoption: Clinical readiness still requires thorough geometric testing, additional anatomical and magnetic-field studies, and compliance with adopted protocols.
  • CBCT-to-CT for image-guided (adaptive) radiotherapy: CBCT-to-CT conversion improved image quality and speed relative to registration and analytical corrections, supporting dose calculation and image guidance.
  • CBCT-to-CT for image-guided (adaptive) radiotherapy: Clinical CBCT-to-CT use is constrained by differing CBCT and CT fields of view, which can hinder dose calculation for replanning.
  • PET attenuation correction: DL-based sCT methods consistently outperformed commercial MRAC methods, but no reviewed method had yet been clinically implemented.
  • Training strategies: Paired training was slightly better than unpaired training, although differences were not always statistically significant.

V. Conclusion

This review organizes deep learning-based synthetic CT generation around three clinical applications and finds an active, growing field with promising results. Clinical translation still requires broader validation, commissioning, quality assurance, and regulatory compliance.

  • The review covers sCT generation for MR-only radiotherapy planning, CBCT-based adaptive radiotherapy, and PET attenuation-map generation.
  • The authors compare imaging protocols, DL architectures, and performance using commonly reported metrics across the three categories.
  • DL-based sCT generation is an active and growing research area with substantial ongoing work and a promising future.
  • For head-and-neck/brain and pelvis, sCT appears feasible, with radiotherapy dose differences from CT-based planning below 1% and better PET attenuation-correction performance than standard MRAC methods.
  • Clinical adoption requires evaluation of generalisation across multiple centres and comprehensive commissioning and QA methods to support treatment efficacy and patient safety.

Appendix

The appendix documents the database search strategy and inclusion framework used for the review, together with abbreviations for imaging, learning, evaluation, and radiotherapy terms. It also defines the three application categories used to organize the evidence.

  • Search strategy: The search covered PubMed, Scopus, and Web of Science using title, abstract, and keyword fields.
  • Search strategy: The query combined radiotherapy, imaging, healthcare, CBCT, PET, and attenuation-correction terms with synthetic-CT and deep-learning terminology.
  • Search strategy: The search also included alternative terms for convolutional networks, CNNs, GANs, and artificial intelligence.
  • Selection framework: Figure 4 schematizes the review’s inclusion and exclusion criteria, including time window, keywords, article type, content, and the three defined categories.
  • Abbreviations: The abbreviation list covers network components, imaging modalities, radiotherapy measures, PET reconstruction errors, and commonly reported metrics such as MAE, RMSE, DSC, PSNR, and SSIM.
  • Abbreviations: The glossary includes clinical and imaging terms such as CBCT, CT, DL, MR, MRAC, PET, sCT, SUV, UTE, VOI, and µ-map.
Loading 2102.02734v2…