PaperlayerView exact revision on arXiv

Source-linked AI summary

Squaring-the-Sphere: Distortion-Free and Seamless Panorama Generation via CubeAtlas

Xuhui Zuo, Chenjie Cao, Tengfei Wang, Junta Wu, Yisu Zhang, Chunchao Guo

arXiv:2610.31565v1cs.CV

TL;DR

ERP and cubemap representations each leave a challenge for panorama generation: balancing spherical continuity with perspective fidelity. SQUARING-THE-SPHERE uses CubeAtlas and layout-aware sparse attention for joint panorama generation; it reports improved seam continuity and spherical geometry across benchmarks. It achieves the lowest SeamCube on all four benchmarks, reducing cube-edge error by 63–78% relative to CubeDiff.

  • Problem

    Panorama generation must combine global spherical continuity with local perspective fidelity, which ERP distortion and cubemap face-boundary inconsistency make difficult.

    Source: §1 · p. 1

  • Method

    CubeAtlas packs equatorial faces and rotated half-polar regions into one canvas for joint DiT generation, with layout-aware sparse attention and perspective-space super-resolution.

    Source: §1 · pp. 2–3, §3.1.2 · p. 5

  • Results

    63–78%: SQUARING-THE-SPHERE reduces cube-edge error relative to CubeDiff and achieves the lowest SeamCube on all four benchmarks.

    Source: §4.1 · p. 7

  • Takeaways & Limitations

    Across diverse benchmarks, the framework improves seam continuity and spherical geometry while preserving visual fidelity and text adherence.

    Source: §1 · p. 3, §5 · p. 9

    Generating a complete 360° environment from one perspective observation is under-constrained, so content outside the observed FoV must be synthesized rather than faithfully recovered.

    Source: A LIMITATIONS · p. 13

Abstract

from arXiv · show

Generating 360-degree panoramas requires both local perspective fidelity and global spherical continuity. Existing representations face a trade-off: equirectangular projection (ERP) enables seamless panorama modeling but suffers from severe distortion and a domain gap to pre-trained image generators, whereas cubemaps preserve perspective geometry but often produce inconsistent faces and visible seams. We present Squaring-the-Sphere, a panoramic image generation framework built on a novel representation CubeAtlas, which unfolds all cubemap faces into a single compact canvas. By arranging orientation-aligned, rotated polar regions alongside the equatorial faces, this representation preserves the perspective geometry of cubemaps while organizing the cross-face relationships required for seamless panorama assembly. It further enables a pretrained diffusion transformer to model the complete panorama jointly in a single pass, without altering the backbone architecture or positional encoding. While CubeAtlas has a token count comparable to ERP, its structured layout makes many dense attention interactions unnecessary. We therefore introduce a layout-aware sparse attention mask that preserves the interactions required for panorama continuity while removing geometrically irrelevant token pairs. As a result, CubeAtlas achieves better quality than ERP modeling with lower attention computation. Furthermore, we introduce a training-free super-resolution technique and a fusion scheme that enhance fine details in distortion-free perspective space before achieving the final ERP panorama. Experiments on various benchmarks demonstrate that our method improves geometric fidelity, cross-view consistency, and high-resolution visual quality over previous methods.

1 INTRODUCTION

Panorama generation must preserve local perspective fidelity and global spherical continuity, but ERP and cubemaps each compromise one of these goals. SQUARING-THE-SPHERE uses CubeAtlas to jointly model aligned cubemap regions, with structured attention and a layout designed for seamless assembly.

1 INTRODUCTION

  • ERP distorts polar regions and departs from perspective-image priors, while cubemaps preserve perspective geometry but make continuity across face boundaries challenging.The paper identifies unifying global coherence with perspective fidelity as a central problem for high-quality panorama generation.Source: §1 · p. 1
  • The framework targets seamless, geometrically coherent panoramas across indoor and outdoor scenes while combining cubemap perspective geometry with global continuity.Its stated contributions also include evaluation of boundary continuity, regional perceptual quality, geometric distortion, text adherence, and human preference.Source: Figure 1 · p. 2, §1 · p. 3
  • CubeAtlas unfolds the cubemap into one compact canvas, arranging equatorial faces beside rotated, orientation-aligned half-polar regions.Complementary rotated polar copies are later combined to reconstruct complete polar faces.Source: §1 · p. 2, Figure 2 · p. 3
  • A pretrained diffusion transformer jointly models the panorama in one pass, using layout-aware sparse attention to retain continuity-related interactions while masking irrelevant token pairs.The framework reuses existing DiT backbones and positional encodings without modification.Source: §1 · p. 2

2 RELATED WORK

Prior panorama methods use ERP canvases or separate perspective faces, while image-to-panorama systems also differ in how they map perspective inputs to the sphere. SQUARING-THE-SPHERE instead learns joint generation in a packed CubeAtlas representation using a pretrained DiT’s native positional encoding and backbone.

2 RELATED WORK

  • ERP methods facilitate global generation but polar stretching mismatches perspective-image priors; cubemap methods preserve perspective geometry but retain separate face grids and need seam-mitigation strategies.CubeDiff and DreamCube use panorama-specific signals or operators to improve boundary consistency.Source: §2 · p. 3
  • Earlier image-to-panorama methods use camera-aware projection and inpainting, whereas 360Anything and HY-Pano 2.0 [HY-World Team et al., 2026] learn geometry-free perspective-to-ERP mappings.SQUARING-THE-SPHERE also uses geometry-free conditioning, but learns the mapping in CubeAtlas rather than directly generating ERP.Source: §2 · pp. 3–4
  • Training-free adaptations such as SpheRoPE [Hirschorn et al., 2026] and SphereDiff add spherical constraints at inference, while SQUARING-THE-SPHERE fine-tunes a pretrained DiT on CubeAtlas for joint cross-face generation.The method retains the pretrained model’s perspective-image prior and native backbone structure.Source: §2 · pp. 3–4

3 METHOD

SQUARING-THE-SPHERE organizes panorama generation around CubeAtlas, a compact canvas that retains cubemap perspective geometry while connecting faces for joint modeling. The method reconstructs polar faces from rotated regions, then applies sparse attention and perspective-space super-resolution.

3.1.1 PREVIOUS REPRESENTATIONS

  • ERP distorts polar regions and separates continuity across image boundaries, while cubemaps preserve perspective geometry but can develop inconsistent faces and visible seams.CubeAtlas is designed to address this representation trade-off.Source: §3.1.1 · pp. 4–5

3.1.2 CUBEATLAS

  • CubeAtlas places the four equatorial faces in a central band and aligns rotated polar halves with each face to organize cross-face relationships on one canvas.The equator-adjacent halves provide local context for neighboring equatorial faces.Source: §3.1.2 · p. 5
  • At 2S × 4S × C, the half-polar canvas matches ERP latent size while retaining cubemap perspective geometry and avoiding redundant polar tokens.A copied left-edge context strip is appended before VAE encoding to represent horizontal periodicity.Source: §3.1.2 · p. 5
  • Rotated polar halves are converted into complementary front–back and left–right estimates, which are joined to reconstruct each complete polar face.The generated CubeAtlas is converted to a cubemap and then ERP for standard rendering, storage, and evaluation.Source: §3.1.2 · p. 5
  • Direction-aware blending favors the front–back estimate where |v| > |u| and the left–right estimate where |u| > |v|, smoothing transitions near the diagonals.The fusion preserves directional consistency across the polar observations before the reconstructed faces are reprojected to ERP.Source: §3.1.2 · pp. 5–6

3.2 MODEL DESIGN

  • The model directly conditions on the input image and uses a pretrained DiT with a layout-aware mask that retains 75% of dense target-to-target attention connectivity.Text and conditioning tokens remain globally connected to target tokens; cross-latitude communication can pass through the equatorial band across DiT layers.Source: §3.2 · p. 6
  • A frozen 4× Pixel Diffusion Decoder enhances overlapping cubemap views in perspective space, blends their overlaps, and corrects low-frequency shifts against the base panorama.The process produces 10k ERP panoramas while retaining generated high-frequency details.Source: §3.2 · pp. 6–7

4 EXPERIMENTS

Across benchmarks, SQUARING-THE-SPHERE improves seam continuity and spherical geometry while maintaining visual fidelity and text adherence. Ablations show that CubeAtlas addresses the ERP–cubemap trade-off, sparse attention lowers attention cost, and perspective-space super-resolution preserves cross-face coherence.

4.1 MAIN RESULTS

  • SQUARING-THE-SPHERE achieves the lowest SeamCube on all four benchmarks, leads Eq-GI and Pole-GI on every benchmark, and improves visual fidelity and text adherence.It also achieves the best CLIP-FID on all three public benchmarks, leads FID and KID on RealSee3D and FID on Laval-Indoor, and obtains the highest TextAdh.Source: §4.1 · p. 7, Table 1 · p. 8
  • In qualitative comparisons, SQUARING-THE-SPHERE maintains coherent geometry and lighting across panorama and cubemap boundaries while reducing polar deformation seen in direct ERP generation.The comparison covers CubeDiff, DreamCube, HunyuanPano, and HY-Pano-2.0 across diverse indoor and outdoor scenes.Source: §4.1 · pp. 7–8

4.2 ABLATION

  • With backbone and training settings fixed, CubeAtlas reduces cubemap face discontinuities and ERP polar deformation while achieving the strongest Eq-IQ and Pole-IQ and competitive distortion scores.Comparisons use base-resolution outputs without super-resolution or additional restoration.Source: §4.2 · p. 8, Figure 6 · p. 9
  • Layout-aware sparse attention preserves comparable overall quality while reducing unnecessary target-to-target interactions and attention cost without changing CubeAtlas’s token count.It reduces SeamCube and improves Eq-Aes, Eq-GI, and Pole-GI, with minor degradations in Eq-IQ and Pole-IQ and a modest increase in SeamERP; CubeAtlas remains more efficient than ERP at the same token count.Source: §4.2 · pp. 8–9, Table 2 · p. 8
  • The full perspective-space super-resolution pipeline recovers sharper structures and textures while maintaining panorama-wide coherence.ERP-based enhancement is uneven, while cubemap processing without overlap fusion introduces boundary inconsistencies.Source: §4.2 · p. 9, Figure 7 · p. 9

5 CONCLUSION

The conclusion presents SQUARING-THE-SPHERE as a representation-driven framework for text-guided image-to-panorama generation. It reports improved geometry, seam continuity, visual fidelity, and text adherence, and discloses the use of generative AI tools in preparing and evaluating the work.

5 CONCLUSION

  • SQUARING-THE-SPHERE combines CubeAtlas, layout-aware sparse attention, geometry-free conditioning, and perspective-space super-resolution for text-guided image-to-panorama generation.The conclusion describes CubeAtlas as jointly generating equatorial faces and rotated half-polar regions while retaining an ERP-like token count.Source: §5 · p. 9
  • Across diverse benchmarks, the framework improves seam continuity and spherical geometry while preserving visual fidelity and text adherence.The conclusion identifies panorama representation as a route for adapting generative image models to omnidirectional synthesis.Source: §5 · p. 9
  • The authors used generative AI tools for manuscript editing, training and evaluation captions, and automated evaluation, and reviewed the AI-assisted work.They state that they take responsibility for the final content.Source: AI USE STATEMENT · p. 10

A LIMITATIONS

The current model is trained and evaluated at a fixed base resolution and does not explicitly support arbitrary-resolution generation. The task also involves uncertain content beyond the observed field of view, and the study covers static panoramas.

A LIMITATIONS

  • The model is trained and evaluated at a fixed base resolution and does not explicitly support arbitrary-resolution generation.The perspective-space super-resolution pipeline produces panoramas up to 5120 × 10240, which the authors say is sufficient for most downstream applications.Source: A LIMITATIONS · pp. 12–13
  • Generating a complete 360° environment from one perspective observation is under-constrained because unseen surroundings must be synthesized and multiple panoramas may fit the same input.The authors also report that vague or incomplete text descriptions may lead to less coherent or less controllable generations.Source: A LIMITATIONS · p. 13
  • The study focuses on static panoramas; temporally consistent panoramic video and interactive scene generation remain future directions.Source: A LIMITATIONS · p. 13

B SUPPLEMENTARY IMPLEMENTATION DETAILS

The supplementary details describe the training and evaluation data, conditioning setup, optimization, and perspective-space super-resolution pipeline. A separate forced-choice study compares base-resolution outputs against four competing methods.

B SUPPLEMENTARY IMPLEMENTATION DETAILS

  • The training corpus combines approximately 50K public, 20K internally collected, and 5K synthetic panoramas, sampled at an approximate real-to-synthetic ratio of 6:4.The evaluation suite includes RealSee3D, Laval-Indoor, SUN360, and a self-collected set; the latter has no ground-truth panorama and uses reference-free metrics.Source: B SUPPLEMENTARY IMPLEMENTATION DETAILS · p. 13
  • The training targets use a 32-pixel copied edge strip for wrap-around seam context, which is removed after decoded padding is used for boundary blending.Sparse attention applies only to target-to-target interactions; text and image-conditioning tokens remain globally connected.Source: B SUPPLEMENTARY IMPLEMENTATION DETAILS · p. 13
  • For each training panorama, perspective conditioning samples FoV uniformly from 50°–90° and pitch from −75°–75°, with yaw and roll fixed at 0°; captions describe the complete panorama at randomly selected concise or detailed granularity.Training uses AdamW for 8,000 steps, with a global batch size of 64 and a learning rate of 10⁻⁵.Source: B SUPPLEMENTARY IMPLEMENTATION DETAILS · pp. 13–14
  • The super-resolution stage upsamples six overlapping perspective cubemap views by 4× and linearly blends their reprojected overlaps to suppress cross-view discontinuities.Removing the 128-pixel boundary context yields 2560 × 2560 effective cube faces and a final 5120 × 10240 ERP panorama.Source: B SUPPLEMENTARY IMPLEMENTATION DETAILS · p. 14

C USER STUDY

  • In a two-alternative forced-choice study, SQUARING-THE-SPHERE was preferred over CubeDiff, DreamCube, HunyuanPano, and HY-Pano-2.0 in 96.7%, 99.7%, 85.7%, and 63.0% of comparisons, respectively.Twenty-eight volunteers made 2,800 comparisons in total, using base-resolution outputs without super-resolution.Source: C USER STUDY · p. 14, Figure 8 · p. 14

D EVALUATION METRICS

The evaluation covers distributional fidelity, boundary continuity, regional perceptual and geometric quality, and text adherence. Perspective views support distributional metrics, while a five-point VLM rubric assesses regional quality and prompt matching.

Distributional Fidelity

  • FID, KID, and CLIP-FID are computed on pooled 90° tangent views from 12 icosahedron directions, with lower values indicating better performance.Laval-Indoor excludes views below −23° and uses eight valid directions; distributional metrics are unavailable for the self-collected set without ground-truth panoramas.Source: D EVALUATION METRICS · p. 15

Boundary Continuity

  • ERP wrap-around and cubemap edge errors are reference-free RGB mean absolute error diagnostics, and lower values indicate better boundary continuity.SeamCube evaluates native cubemap faces over 12 shared edge pairs; it is undefined for ERP-only outputs, and both diagnostics are interpreted alongside qualitative comparisons because texture variation also affects them.Source: D EVALUATION METRICS · p. 15

VLM-based Regional Quality

  • A five-point VLM rubric scores equatorial and polar views separately for image quality and geometric integrity, with equatorial views also rated for aesthetics.Regional scores are averaged across views within each panorama and then across panoramas; the distortion criteria cover specified structural, texture, and polar-patch artifacts, not only geometric reconstruction error.Source: D EVALUATION METRICS · pp. 15–16

Text Adherence

  • TextAdh uses the same five-point scale to judge whether the full ERP matches its caption, averaging scores over samples with non-empty prompts.The judge considers objects, attributes, counts, and spatial relations, with scores ranging from all described elements present and correct to a scene nearly unrelated to the prompt.Source: D EVALUATION METRICS · p. 16

VLM-based Regional Quality

  • Figure 9 shows VLM rationales and scores for selected regional views, while TextAdh evaluates the full panorama.Its examples proceed from equatorial quality to polar quality and text adherence.Source: D EVALUATION METRICS · p. 16

A. Equatorial Evaluation

The equatorial evaluation asks a VLM to rate four eye-level perspective views separately for image quality, aesthetics, and distortion.

Equatorial Evaluation

  • Each panorama is shown as a strip of Front, Right, Back, and Left views, each scored separately on a five-point scale.The distortion criterion asks whether a view resembles a normal-camera photo; wavy lines, broken objects, and moiré or banding lower the score.Source: A. Equatorial Evaluation · p. 16
  • The evaluator gives a brief reason before each score for image quality, aesthetics, and distortion..Source: A. Equatorial Evaluation · p. 16
  • The scale runs from Excellent (5) to Awful (1), with Good (4), Fair (3), and Poor (2) between them..Source: A. Equatorial Evaluation · p. 16

B. Polar Evaluation

The evaluation uses close-up polar views and full-panorama prompt matching to assess distinct aspects of generated panoramas. It also checks VLM scores against human judgments across six criteria.

B. Polar Evaluation

  • Polar evaluation scores Zenith and Nadir close-ups separately for image quality and distortion on a five-point scale.The distortion prompt names defects including swirl, central black holes, radial seams, warped geometry, smearing, and nadir camera or tripod patches; the full ERP is provided only for context.Source: B. Polar Evaluation · p. 16
  • Figure 9 gives VLM rationales and scores for Front and Zenith examples; “Distortion” denotes geometric integrity, with higher scores indicating fewer artifacts.Equatorial views are rearranged for display, and the input caption is excerpted.Source: Figure 9 · p. 17

C. Text Adherence

  • Text adherence judges whether visible content in the full ERP matches the prompt, from all described elements correct (5) to a nearly unrelated scene (1).The scale also distinguishes minor omissions, several missing or incorrect elements, and most of the description unsatisfied.Source: C. Text Adherence · p. 17

B. Polar Evaluation

  • Table 4 compares full-polar and half-polar packing and identifies the layout-aware sparse-attention condition as “+ Sparse Attn.”Padding tokens are omitted from the token-length count for simplification.Source: Table 4 · p. 18
  • Across 30 panoramas, VLM scores correlate with human judgments on all six criteria, with Spearman’s ρ ranging from 0.7698 to 0.9358.Three blinded raters scored each panorama, and their mean opinion scores were compared with panorama-level VLM scores.Source: Table 3 · p. 17, C. Text Adherence · p. 18

E ADDITIONAL RESULTS

The additional results favor half-polar packing over full-polar packing on quality and efficiency. They also show geometry-free generation from perspective views with varied fields of view and directions.

E.1 FULL-POLAR VS HALF-POLAR PACKING

  • Full-polar packing increases token count by 50% and computational cost by approximately 37%, yet yields no clear quality improvement over half-polar packing.The half-polar design retains sufficient polar context while avoiding redundant complete polar copies, and is adopted for the final model; Table 4 evaluates quality and efficiency on the self-collected benchmark.Source: §E.1 · p. 18

E.2 GEOMETRY-FREE GENERATION FROM ARBITRARY VIEWS

  • SQUARING-THE-SPHERE generates panoramas from perspective views with varied fields of view and directions, including upward- and downward-looking inputs, without camera calibration.It learns the input view’s panorama placement and synthesizes surrounding content; red outlines mark the corresponding regions in generated ERPs.Source: §E.2 · p. 18

E.3 ADDITIONAL QUALITATIVE RESULTS

Additional qualitative results broaden the evaluation across datasets, super-resolution strategies, viewing directions, and scene styles, highlighting cross-view consistency and high-resolution detail synthesis.

E.3 ADDITIONAL QUALITATIVE RESULTS

  • Comparisons span Laval-Indoor, SUN360, and RealSee3D across SQUARING-THE-SPHERE, HY-Pano-2.0, HunyuanPano, DreamCube, and CubeDiff.The examples include diverse indoor and outdoor scenes.Source: §E.3 · p. 18, Figure 10 · p. 19
  • The high-resolution examples compare native sampling, ERP-domain super-resolution, perspective-space enhancement without overlap fusion, and the complete pipeline.Enlarged crops compare local detail recovery and cross-view consistency.Source: §E.3 · p. 18, Figure 11 · p. 19
  • Further generations span realistic, cinematic, stylized, and illustrative appearances while presenting diverse indoor and outdoor scenes.The examples show conditioning content and visual style preserved alongside coherent structures throughout the 360° field of view.Source: §E.3 · p. 18, Figure 13 · p. 21
  • Generated ERP panoramas accommodate downward- and upward-looking perspective inputs without camera calibration or explicit input projection.Red outlines mark the corresponding input regions in the generated panoramas.Source: Figure 12 · p. 20

Related papers

Loading 2610.31565v1…