PaperlayerView exact revision on arXiv

Source-linked AI summary

SparseTokenGS: Gaussian-based Sparse-View CBCT Reconstruction via Reusable Token Pool

Zihan Deng, Zhenzhen Gong, Yujie Liu, Jiaxin Gong, Dingkun Liu, Kexian Xie, Yixing Huang, Peng Jin, Tianye Niu, Zhisheng Wang, Fenglin Liu, Hongming Shan, Lequan Yu

arXiv:2610.38360v1cs.CVcs.AI

TL;DR

Sparse-view reconstruction makes it difficult to preserve fine detail and consistency when scan-wide appearance is unavailable during coordinate decoding. SparseTokenGS reuses a token pool at anchors and query coordinates in one forward pass. It outperforms compared feed-forward methods on three public datasets, while six-view clinical reconstruction supports tooth-and-pulp separation for lesion localization.

  • Problem

    After a feed-forward Gaussian field is formed, later coordinate decoding cannot retrieve whole-scan appearance from local features, making fine detail and volume consistency difficult to preserve together.

    Source: §I · p. 2

  • Method

    SparseTokenGS uses one reusable scan-specific token pool at both Gaussian anchors and query coordinates to condition the reconstructed volume in one forward pass.

    Source: §I · p. 2, A. Dual-Level Token Conditioning · p. 3

  • Results

    SparseTokenGS outperforms compared feed-forward methods in image quality from three to eighteen views on three public datasets, and six-view clinical reconstruction supports tooth-and-pulp separation for lesion localization.

    Source: §VII · p. 10

  • Takeaways & Limitations

    The reusable pool provides a route for feed-forward methods to retain scan-level memory through coordinate decoding, with clinical dental results supporting lesion localization.

    Source: §VII · p. 10

Abstract

from arXiv · show

Sparse-view cone-beam computed tomography is widely used for fast, low-dose volumetric imaging and lesion localization in dentistry and image-guided interventions, but it introduces severe artifacts. Feed-forward Gaussian methods transfer the reconstruction cost to training and decode each coordinate from projection-aligned features and neighboring anchors. However, information from the whole scan is written only when the field is constructed and cannot be retrieved during coordinate decoding, leaving the volume inconsistent within the scan. This is a common limitation of such methods: once the field is fixed, decoding uses only local features and reconstruction quality drops substantially. We propose SparseTokenGS, a Gaussian-based method for sparse-view cone-beam computed tomography reconstruction via a reusable token pool. Anchor Token Fusion and Point-Token Attention query this shared pool at the anchors and at the query coordinates, respectively. The local Gaussian prior is retained, and the volume is reconstructed in one forward pass, without per-scan optimization or a projection-domain loss. On three public chest and dental datasets, SparseTokenGS achieves the highest reconstruction performance among the compared methods. On clinical dental scans, reconstruction from only six projections reaches 29.51 decibels in peak signal-to-noise ratio and a structural similarity of 0.842 against the full-view reference, and off-the-shelf tooth-and-pulp segmentation keeps these structures separable for root-canal localization. Experiments on volcanic rock and chicken show performance after same-object adaptation. A reusable token pool gives feed-forward reconstruction a mechanism that keeps scan-wide context available through decoding and can benefit future rapid clinical lesion localization and analysis. The code is available at: https://github.com/FrankDengAI/SparseTokenGS.

I. Introduction

Sparse-view CBCT can speed imaging and reduce radiation, but sparse angular sampling makes reconstruction underdetermined and leaves streaks. SparseTokenGS addresses the loss of scan-wide context during coordinate decoding by reusing a token pool at anchors and query coordinates.

I. Introduction

  • Sparse angular sampling makes CBCT reconstruction underdetermined; FDK leaves streak artifacts, while 3D iterative methods are slow and can remove low-contrast texture.Sparse protocols are used for fast lesion localization, shorter reconstruction time, or lower radiation burden.Source: §I · p. 1
  • Feed-forward methods amortize reconstruction over training, but write scan-level context only while constructing the field.Afterward, coordinate decoding relies mainly on local features and cannot retrieve whole-scan appearance, making detail and volume consistency difficult to preserve together.Source: §I · pp. 1–2
  • SparseTokenGS reuses one pose-free scan-level token pool to condition both Gaussian anchors and query coordinates during decoding.Anchor Token Fusion adds a zero-initialized residual; Point-Token Attention reads the same pool alongside view and neighboring Gaussian features.Source: §I · p. 2
  • One forward pass returns the volume using volumetric supervision alone, without projection loss or per-scan optimization.The design retains local projection features for anchor geometry while using the pool to condition appearance.Source: §I · p. 2
  • Clinical dental evaluation, ablations, and volcanic rock and chicken experiments assess localization support, token use, and generalization.An off-the-shelf tooth-and-pulp segmenter separates the structures on held-out clinical dental scans.Source: §I · p. 2

A. Learning-based Reconstruction

Learning-based reconstruction includes image-domain correction, projection-informed methods, and continuous Gaussian or neural-field representations. SparseTokenGS uses a reusable scan-level pool so later coordinate queries can retrieve context that prior feed-forward fields leave in local features.

A. Learning-based Reconstruction

  • Image-domain correction is low-cost but starts from streaked FDK volumes, while projection-image methods unroll solver steps and can depend on training geometry.Sampling-diffusion reconstruction remains iterative; discrete-grid methods also include coordinate-based projection placement and crossscale pretraining.Source: A. Learning-based Reconstruction · p. 2

B. Neural Fields, Gaussians, and Tokens

  • Neural fields and Gaussians represent volumes continuously or with primitives, reducing computation and strengthening local structure, but scan-fitted methods require per-scan optimization.Feed-forward Gaussian and triplane methods write scan appearance during field construction, leaving later queries with local features rather than reusable tokens.Source: B. Neural Fields, Gaussians, and Tokens · pp. 2–3

III. Methodology

  • SparseTokenGS retains a reusable scan-level token pool until decoding, with reads at Gaussian anchors and query coordinates.The methodology also covers pool construction, anchor parameterization, and a hybrid query trained with volumetric supervision alone.Source: §III · p. 3

A. Dual-Level Token Conditioning

SparseTokenGS reads a shared scan-specific token pool at both anchors and query coordinates, while retaining local Gaussian features. The pool is built from sampled multi-view features and refined by attention.

A. Dual-Level Token Conditioning

  • The shared encoder produces a full-resolution map for view alignment and a bottleneck map for anchors and the token pool.At anchors, cross-attention reads the pool and adds a zero-initialized residual to local features; at query coordinates, a separate attention block supplies a contextual feature without neighbor aggregation.Source: A. Dual-Level Token Conditioning · p. 3

B. Scan-Specific Token Pool

  • The scan-specific appearance memory is formed from flattened bottleneck features, retaining at most 2048 uniformly spaced rows without camera pose, view index, or 2D positional embeddings.A learned 512-token matrix queries this memory, then three decoder blocks apply cross-attention, self-attention, and feed-forward residual updates.Source: B. Scan-Specific Token Pool · p. 4
  • The pool construction samples multi-view feature maps, then refines tokens through cross-attention to sampled keys and values followed by token self-attention.The pool length is fixed independently of view count and output points.Source: B. Scan-Specific Token Pool · p. 4

C. Token-Conditioned Gaussian Anchors

SparseTokenGS places anchors on a 14^3 lattice, predicts their features and Gaussian geometry from view-aggregated local evidence, and uses the token pool to condition anchor appearance. Geometry remains tied to local projection features, with bounded center offsets and scale components.

C. Token-Conditioned Gaussian Anchors

  • At each of 2744 lattice nodes, channelwise maximum over views aggregates bottleneck samples; separate MLPs predict anchor features and ten geometry channels.The geometry channels encode a 3D offset, 3D scale, and quaternion.Source: C. Token-Conditioned Gaussian Anchors · p. 4
  • The center offset is bounded to half a lattice step, while each scale component lies between 0.7R^-1/2 and 1.3R^-1/2.The quaternion is normalized and defines the rotation used to form the covariance.Source: C. Token-Conditioned Gaussian Anchors · p. 4
  • The offset bound constrains the Gaussian center, not its support, so neighboring kernels may overlap.The offset and scale logits are multiplied by τ = 512 before the sigmoid.Source: C. Token-Conditioned Gaussian Anchors · p. 4
  • Anchor Token Fusion adds a zero-initialized residual to local anchor features, while the geometry head receives no token and remains tied to local projection evidence.The zero initialization makes fused features initially equal the local prediction.Source: C. Token-Conditioned Gaussian Anchors · p. 4

D. Hybrid Point Query

The hybrid query combines view-aligned evidence, features from five nearby Gaussian anchors, and a coordinate-dependent read from the shared token pool. Volumetric training uses sampled-intensity losses and no projection-consistency term.

D. Hybrid Point Query

  • The decoder combines three 128-dimensional features: view-aligned evidence, an aggregated Gaussian feature, and a coordinate-dependent token feature.The view branch takes a channelwise maximum over views, and the Gaussian neighborhood uses the five nearest unshifted lattice nodes.Source: D. Hybrid Point Query · p. 4
  • The five nearest unshifted lattice nodes are selected on the original lattice, while Gaussian density weights use the shifted centers.The selected weights are not normalized, retaining kernel magnitude in the aggregated feature.Source: D. Hybrid Point Query · p. 4
  • The coordinate MLP forms a query for Point-Token Attention, so each coordinate reads the shared pool with attention weights that can vary by location.This token feature supplements the two local branches.Source: D. Hybrid Point Query · p. 4
  • A pointwise-convolution decoder maps the 384-channel concatenated feature to the predicted intensity, using skip connections into later stages.At evaluation, predictions are clipped to [0, 1].Source: D. Hybrid Point Query · p. 4
  • Training uses a weighted combination of MSE and mean absolute error with a sample-order windowed similarity term, but no projection-consistency loss.The similarity term is distinct from spatial SSIM computed on reconstructed volumes at evaluation.Source: D. Hybrid Point Query · p. 5

IV. Experimental Setting

The experiments use simulated chest and dental datasets with controlled projection counts, and compare reconstruction metrics against iterative, image-domain, and feed-forward methods.

A. Simulated Experiments and Datasets

  • Chest LUNA16 and dental ToothFairy and ToothFairy4 scans are split about 8:1:1, normalized, resampled to 256^3, and used to simulate cone-beam projections.LUNA25 is also shown with the cohorts in Fig. 8.Source: A. Simulated Experiments and Datasets · p. 5
  • Models are trained at 3, 6, 10, 14, and 18 views sampled from a fixed 180° arc; training offsets vary, while test offsets are fixed at zero.Each dataset uses 50 projections along the arc.Source: A. Simulated Experiments and Datasets · p. 5

B. Baselines and Evaluation Protocol

  • Comparisons include FDK, SART, FBPConvNet, DIF-Net, feed-forward DIF-Gaussian without test-time refinement, and X-GRM; PSNR, SSIM, and RMSE are measured with data range 1.The learned feed-forward baselines are trained separately for each dataset and view count.Source: B. Baselines and Evaluation Protocol · p. 5

C. Clinical Data and Other Objects

This block covers the evaluation cohorts, implementation and computational cost, and simulated reconstruction results. At six views, SparseTokenGS exceeds DIF-Gaussian on all three reported datasets.

C. Clinical Data and Other Objects

  • Simulated chest and dental volumes were resampled to 256^3; volcanic rock and chicken were separately evaluated using eighteen views and same-specimen adaptation.Source: C. Clinical Data and Other Objects · p. 5

D. Implementation and Computational Cost

A. Main Results

B. Ablation Results

The ablations compare token reads, the similarity term, and anchor-grid resolution. Removing the point-level read causes the largest reported PSNR drop.

B. Ablation Results

C. Clinical Dental Evaluation

On held-out clinical dental scans, SparseTokenGS reconstruction retains dental structures that an off-the-shelf segmenter delineates without retraining. The reported segmentation supports root-canal localization.

C. Clinical Dental Evaluation

D. Generalization to Different Objects

After same-object adaptation, SparseTokenGS is compared on volcanic rock and chicken reconstructions. Its reported PSNR and SSIM exceed the other listed methods on both specimens.

D. Generalization to Different Objects

A. What Dual-Level Conditioning Adds

B. Interpreting the Reconstruction Evidence

SparseTokenGS leads the compared entries across datasets and view counts, while ablations and clinical scans examine the token pool’s role and its practical scope. Same-object adaptation also supports results on rock and chicken, and the conclusion identifies rapid clinical lesion localization as future work.

B. Interpreting the Reconstruction Evidence

  • SparseTokenGS attains the highest PSNR and SSIM among compared entries on every dataset and view count, with its advantage over feed-forward baselines persisting from three to eighteen views.Each view count uses its own checkpoint while the pool length, decoder, and anchor grid remain fixed.Source: B. Interpreting the Reconstruction… · p. 9
  • Ablations place the full checkpoint above variants removing anchor fusion, the token pool, or the similarity term; removing the point-level read causes the largest stated gap.The 143-anchor grid has the highest validation score, balancing field coverage against anchor learnability.Source: B. Interpreting the Reconstruction… · p. 9

C. Clinical Interpretation and Other Objects

  • On six-view clinical dental scans, reconstruction reaches 29.51 dB against the scanner reference, while off-the-shelf TIPs separates tooth and pulp without retraining.Pulp Dice exceeds 50% on the three displayed cases; larger absolute errors remain around tooth and cortical boundaries where few angles underdetermine edges.Source: C. Clinical Interpretation and Other… · p. 9
  • On rock and chicken, short same-object adaptation updates the encoder and attention weights, after which the rewritten pool organizes appearance for the new object without architectural redesign.The pool is rewritten from the current scan, while the encoder and attention weights come from the training distribution.Source: C. Clinical Interpretation and Other… · pp. 9–10

VII. Conclusion

  • The conclusion presents reusable scan-level memory through decoding as a route for future feed-forward methods and identifies rapid clinical lesion localization as future work.The paper reports image-quality advantages over compared feed-forward methods from three to eighteen views on three public datasets.Source: §VII · p. 10

Related papers

Loading 2610.38360v1…