Source-linked AI summary
X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks
Xingde Ying, Heng Guo, Kai Ma, Jian Wu, Zhengxin Weng, Yefeng Zheng
TL;DR
The paper addresses CT reconstruction from limited X-ray views without the hundreds of projections and full rotational scan required by conventional methods. It proposes X2CT-GAN, which maps two orthogonal X-rays to a 3D CT volume using a specialized generator, feature fusion, and combined losses. Experiments show improved reconstruction with biplanar inputs, while the method remains intended for niche applications because small anatomies can contain artifacts.
Problem
Conventional CT reconstruction requires hundreds of X-ray projections during a full rotational scan, while reconstructing 3D internal structure from limited X-rays is highly ambiguous.
Method
X2CT-GAN uses a specialized generator, feature fusion, and reconstruction, projection, and adversarial losses to reconstruct 3D CT from two orthogonal X-rays.
Results
Biplanar inputs improve reconstruction accuracy by about 4 dB in PSNR for both X2CT-CNN and X2CT-GAN versus single-X-ray input, while X2CT-GAN improves visual quality.
Takeaways & Limitations
The method may enhance low-cost X-ray machines with CT-like 3D volumes for niche applications such as organ measurement, radiation-therapy dose planning, and surgical guidance.
Takeaways & Limitations
The method is not intended to replace CT, and small anatomies still suffer from artifacts despite accurate reconstruction of general structure.
Abstract
from arXiv · showhide
Computed tomography (CT) can provide a 3D view of the patient's internal organs, facilitating disease diagnosis, but it incurs more radiation dose to a patient and a CT scanner is much more cost prohibitive than an X-ray machine too. Traditional CT reconstruction methods require hundreds of X-ray projections through a full rotational scan of the body, which cannot be performed on a typical X-ray machine. In this work, we propose to reconstruct CT from two orthogonal X-rays using the generative adversarial network (GAN) framework. A specially designed generator network is exploited to increase data dimension from 2D (X-rays) to 3D (CT), which is not addressed in previous research of GAN. A novel feature fusion method is proposed to combine information from two X-rays.The mean squared error (MSE) loss and adversarial loss are combined to train the generator, resulting in a high-quality CT volume both visually and quantitatively. Extensive experiments on a publicly available chest CT dataset demonstrate the effectiveness of the proposed method. It could be a nice enhancement of a low-cost X-ray machine to provide physicians a CT-like 3D volume in several niche applications.
1. Introduction
X2CT-GAN reconstructs a 3D CT volume from two orthogonal X-rays, addressing the limitations of conventional rotational CT acquisition and the ambiguity of 2D-to-3D reconstruction.
- Motivation: CT presents tissues in 3D but requires many rotational X-ray projections, higher radiation dose, and a more costly scanner than standard X-ray imaging.Typical CT reconstruction uses at least 100 images captured during a full rotation, which typical X-ray machines cannot perform.
- Problem and approach: The proposed task reconstructs CT from biplanar X-rays captured in two orthogonal view planes, whose complementary information reduces single-view ambiguity.The authors report a 4 dB PSNR improvement for biplanar over single-view reconstruction in a comparison experiment.
- Method: The method trains on synthesized X-rays generated from public chest CT datasets and uses CycleGAN to transfer real X-rays toward the synthesized style.This enables a network trained with synthesized X-rays to reconstruct CT from real X-rays.
- Method: X2CT-GAN uses a specialized 2D-to-3D generator and novel feature fusion to combine information from the two X-ray views.The work identifies 2D-to-3D mapping and information fusion across views as central design challenges.
- Contribution: Compared with visible-light reconstruction algorithms, the X-ray-based approach can reconstruct both surface and internal structures.The authors position this as a contribution of using X-rays for reconstruction.
2. Related Work
Prior work addresses modality transfer and 3D reconstruction from projections, but X2CT-GAN targets anatomically accurate 3D CT-like volumes from biplanar X-rays without requiring hundreds of rotational projections.
- Cross-Modality Transfer: Cross-modality transfer methods typically preserve input and output dimensionality, whereas X2CT-GAN transfers 2D X-rays to a 3D volume.The paper identifies this dimensional increase as a central distinction from prior medical imaging transfer work.
- 3D Model Extraction from 2D Projections: Earlier 3D extraction methods from X-rays reconstructed bones, rib cages, or lungs using statistical shape models or other prior knowledge.These methods focus on selected structures rather than generating a 3D CT-like image.
- 3D Model Extraction from 2D Projections: Shape-model approaches may produce precise models, but extending them to other organs requires elaborate geometric models prepared in advance.This requirement limits their application scenarios according to the paper.
- CT Reconstruction from X-ray: Classical CT reconstruction uses hundreds of X-rays acquired during a full rotational scan of the body.Filtered back projection and iterative reconstruction are cited as representative methods.
- CT Reconstruction from X-ray: Prior deep-learning work used X-ray sinograms, addressed limited-angle compensation, or predicted 3D objects from a single-image tomography.The paper contrasts these settings with human-readable biplanar X-rays and argues that an additional orthogonal view provides more reconstruction constraints.
- CT Reconstruction from X-ray: Combining adversarial training with reconstruction constraints is reported to extract finer anatomical structures than earlier blurry reconstructions.Blood vessels inside lungs are given as an example of the finer structures targeted.
3. Objective Functions of X2CT-GAN
X2CT-GAN combines conditional adversarial, voxel-wise reconstruction, and projection losses to produce CT volumes that are both realistic and structurally consistent with biplanar X-rays.
- Adversarial Loss: Conditional LSGAN learns a nonlinear mapping from two orthogonal X-rays to a corresponding CT volume while encouraging realistic details.The least-square adversarial objective replaces logarithmic loss to stabilize adversarial training.
- Reconstruction Loss: The MSE reconstruction loss enforces voxel-wise similarity between the generated CT and the ground-truth volume.This constraint addresses the need for high precision in internal 3D structures.
- Projection Loss: The projection loss matches axial, coronal, and sagittal projections of the predicted and ground-truth volumes to enforce general shape consistency.L1 distance is used to encourage sharper image boundaries.
- Total Objective: The final objective combines adversarial, reconstruction, and projection losses with λ1 = 0.1 and λ2 = λ3 = 10.These weights prioritize global shape consistency while retaining adversarial pressure for local realism.
- Connection Types: Connection-C is designed to fuse information from two different views within the objective-function section’s associated connection design.The figure identifies Connection-A and Connection-B as dimensionality-increasing connections.
4. Network Architecture of X2CT-GAN
X2CT-GAN uses parallel 2D-to-3D encoder-decoder pathways for PA and lateral X-rays, followed by a fusion decoder that reconstructs the CT volume.
- Generator: The generator contains parallel encoder-decoder networks for PA and lateral X-rays plus a fusion network that outputs the reconstructed 3D CT volume.The two view-specific pathways extract features before the third decoder combines them.
- Generator: Dense modules in each encoding path extract multilevel information from the input X-ray while compressing feature channels.Each module includes down-sampling, dense convolution, and channel-compressing blocks.
- 2D–3D Connections: Connection-A bridges the 2D encoder and 3D decoder through fully connected layers that reshape flattened features into 3D.Because this conversion loses spatial information, it is used only between the final encoder and first decoder layers.
- 2D–3D Connections: Connection-B provides the remaining 2D-to-3D skip connections by transporting low-level encoder features to the decoder.It is designed to preserve more spatial information than the fully connected conversion.
- Discriminator: A 3DPatchDiscriminator uses 3D convolutional modules to distinguish real from fake CT volumes.The discriminator follows a PatchGAN-like design with instance normalization and ReLU modules.
- Training: The generator and discriminator are trained alternately with Adam for 100 epochs, using learning-rate decay after 50 epochs.The batch size is one because of GPU memory limits.
5. Experiments
Experiments compare X2CT-GAN with MSE-only and baseline reconstruction methods using synthetic and real-world X-rays. Results show benefits from biplanar input, adversarial training, and proposed connection modules.
- Experimental Setup: The augmented LIDC-IDRI dataset evaluates reconstruction with PSNR and SSIM against a reproduced 2DCNN baseline.X2CT-CNN uses MSE loss alone, whereas X2CT-GAN uses the total objective.
- Qualitative Results: X2CT-GAN preserves small anatomical structures better than 2DCNN and X2CT-CNN, which produce blurrier reconstructed volumes.Biplanar-trained models outperform single-view models visually, while single-view X2CT-CNN gives sharper large-organ boundaries than 2DCNN.
- Quantitative Results: Biplanar inputs improve reconstruction accuracy by about 4 dB in PSNR for both X2CT-CNN and X2CT-GAN versus single-view inputs.The comparison attributes the improvement to complementary information from the two orthogonal views.
- Quantitative Results: X2CT-GAN+B decreases PSNR by only 1.1 dB relative to X2CT-CNN+B while substantially improving visual image quality.The combined voxel-level MSE and semantic-level adversarial losses provide a trade-off between quantitative and visual quality.
- Ablation Study: The biplanar model with the combined Connection-B and Connection-C module surpasses other connection combinations in both PSNR and SSIM.Biplanar input also exceeds single-view input without skip connections because the views provide complementary information.
- Ablation Study: Adversarial loss alone performs poorly on PSNR and SSIM, while adding reconstruction loss produces the most significant improvement; projection and conditional losses add slight gains.The GAN framework is therefore evaluated through multiple loss settings rather than adversarial loss alone.
- Real-World Evaluation: On real-world X-rays, qualitative results show plausible reconstructed lung and surface structures despite the model being trained on synthetic data.Because corresponding 3D CT volumes are unavailable for real X-rays, only qualitative evaluation is conducted.
6. Conclusions
The paper reconstructs 3D CT scans end to end from biplanar 2D X-rays using a GAN framework with multiple losses and a 2D-to-3D generator. Experiments show biplanar X-rays outperform single-view X-rays, while future work will assess clinical value.
- Conclusions: The method combines reconstruction, projection, and adversarial losses with a generator that increases data dimensionality from 2D X-rays to 3D CT.The approach is designed for end-to-end reconstruction from biplanar inputs.
- Conclusions: Experiments qualitatively and quantitatively demonstrate that biplanar X-rays are superior to single-view X-rays for 3D reconstruction.The paper identifies clinical evaluation of organ measurement and radiation-therapy dose planning as future work.
7. Appendix
The appendix provides additional qualitative reconstructions across subjects and evaluates real-world X-ray inputs after CycleGAN style transfer. It presents axial slices and 3D renderings for visual comparison.
- Additional Visual Results: Additional results include four CT slices from four subjects and 3D renderings from two subjects using different reconstruction approaches.The appendix compares reconstruction outputs with corresponding ground truths and across methods.
- Qualitative Comparison: Biplanar-input models are more accurate in lateral view, while GAN-based models extract finer anatomical structures such as blood vessels inside lungs.These qualitative comparisons are shown through additional 3D CT renderings.
- Real-World X-Rays: For real-world evaluation, the first columns show two-view real X-rays, the next columns show CycleGAN-transformed X-rays, and the final columns show reconstructed CT renderings.The renderings include PA and lateral views as well as internal structures and surfaces.