PaperlayerView exact revision on arXiv

Source-linked AI summary

DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering

Wei Cheng, Ruixiang Chen, Wanqi Yin, Siming Fan, Keyu Chen, Honglin He, Huiwen Luo, Zhongang Cai, Jingbo Wang, Yang Gao, Zhengming Yu, Zhengyu Lin, Daxuan Ren, Lei Yang, Ziwei Liu, Chen Change Loy, Chen Qian, Wayne Wu, Dahua Lin, Bo Dai, Kwan-Yee Lin

arXiv:2307.10173v2cs.CV

TL;DR

Human-centric rendering datasets have limited diversity and realism, although both matter for rendering quality. DNA-Rendering contributes a large-scale multiview performance dataset and benchmarks across three rendering tasks. Its benchmark findings identify hard deformation and interaction as persistent challenges for current methods.

  • Problem

    Existing human-centric rendering datasets have limited diversity or realism, even though these factors are important to rendering quality.

    Source: §1 · p. 2

  • Method

    DNA-Rendering provides a large-scale multiview human-performance dataset and benchmarks novel view synthesis, novel pose animation, and novel identity rendering.

    Source: §1 · pp. 2–3

  • Results

    Current methods handle hard motion and textures relatively reasonably but remain challenged by hard deformation and interaction in novel-pose animation.

    Source: §4.3 · pp. 8–9

  • Takeaways & Limitations

    The dataset and benchmarks provide standards and detailed evaluations for three rendering tasks, with observations intended to inform future research.

    Source: §1 · p. 3

    The capture setting uses uniform lighting and a constrained field of view; future work targets varied lighting, multi-person scenes, faster capture, and multisensory data.

    Source: §A.5 · p. 17

Abstract

from arXiv · show

Realistic human-centric rendering plays a key role in both computer vision and computer graphics. Rapid progress has been made in the algorithm aspect over the years, yet existing human-centric rendering datasets and benchmarks are rather impoverished in terms of diversity, which are crucial for rendering effect. Researchers are usually constrained to explore and evaluate a small set of rendering problems on current datasets, while real-world applications require methods to be robust across different scenarios. In this work, we present DNA-Rendering, a large-scale, high-fidelity repository of human performance data for neural actor rendering. DNA-Rendering presents several alluring attributes. First, our dataset contains over 1500 human subjects, 5000 motion sequences, and 67.5M frames' data volume. Second, we provide rich assets for each subject -- 2D/3D human body keypoints, foreground masks, SMPLX models, cloth/accessory materials, multi-view images, and videos. These assets boost the current method's accuracy on downstream rendering tasks. Third, we construct a professional multi-view system to capture data, which contains 60 synchronous cameras with max 4096 x 3000 resolution, 15 fps speed, and stern camera calibration steps, ensuring high-quality resources for task training and evaluation. Along with the dataset, we provide a large-scale and quantitative benchmark in full-scale, with multiple tasks to evaluate the existing progress of novel view synthesis, novel pose animation synthesis, and novel identity rendering methods. In this manuscript, we describe our DNA-Rendering effort as a revealing of new observations, challenges, and future directions to human-centric rendering. The dataset, code, and benchmarks will be publicly available at https://dna-rendering.github.io/

1. Introduction

Human-centric rendering lacks comprehensive datasets that combine diversity with high realism, although these factors are important to rendering quality. DNA-Rendering contributes a large-scale capture dataset, annotations, and benchmarks across three rendering tasks.

1. Introduction

  • Existing human-centric rendering datasets have limited diversity or realism, despite both being important to rendering quality.Capturing high-quality, massive 3D/4D human avatars requires high-end equipment and an efficient processing pipeline.Source: §1 · p. 2
  • DNA-Rendering captures 5000 video sequences and over 67.5M frames across 500 actors, 1500 outfits, and 1187 motion types.Its system uses 60 calibrated RGB cameras and eight synchronized depth sensors, recording up to 4096 × 3000 resolution at 15 fps.Source: §1 · p. 2
  • The project provides frame-level annotations through an automatic pipeline for calibration, color correction, matting, landmark estimation, and SMPLX fitting.The authors report technical refinements intended to make annotations faithful, effective, and efficient.Source: §1 · pp. 2–3
  • Benchmarks evaluate novel view synthesis, novel pose animation, and novel identity rendering across challenging data splits and analyze method robustness and design choices.The reported observations address human-prior robustness, sensitivity to data volume and distribution, and loss effects across rendering metrics.Source: §1 · p. 3
  • The authors position DNA-Rendering as the largest multi-view human-body performance dataset for high-fidelity rendering, with benchmarks and tools for research.They expect the dataset, benchmarks, and tools to support digital-human applications and future research.Source: §1 · p. 3

2. Related Works

Prior human-centric datasets support perception, reconstruction, and rendering research, but often lack the dense views, resolution, diversity, or coverage needed for rendering evaluation. Related work also traces neural implicit representations and approaches to animatable human modeling.

2.1. Human-centric Datasets

  • Perception datasets support tasks such as pose estimation and action recognition, but often lack multiview ground-truth images for rendering evaluation.Human3.6M and MPI-INF-3DHP [48] capture indoor multiview data, while 3DPW captures motion in the wild; limits in samples, views, and resolution constrain rendering evaluation.Source: §2.1 · p. 3
  • Rendering datasets include static scans and dynamic human captures, but many are biased toward standing poses or limited in view density, resolution, or diversity.ZJU-MoCap [56] has ten sequences and 24 cameras under 1K resolution; AIST++ [70, 37] offers varied dance motions but a one-fold scenario and limited view density.Source: §2.1 · p. 3
  • DNA-Rendering contributes the largest high-fidelity multiview dataset for human-centric rendering, with 5000 human performance sequences varying in ethnicity, age, cloth, motion, and interaction.Concurrent works [91] contribute human-avatar datasets focused on detailed human geometry filmed with long-lens cameras; [91] captures at a low frame rate, while has dense multiview capture but only 16 motion sequences.Source: §2.1 · p. 4

2.2. Implicit Neural Body Representation

  • Neural implicit human-rendering methods use representations such as NeRFs and human priors, but prior evaluations may be biased by limited dataset scale and coverage.NeuralBody [56] conditions dynamic radiance fields on body structure and temporal codes; generalizable methods use image features and, in some cases, human priors [87] [49] [13] [34].Source: §2.2 · p. 4

2.3. Animatable Digital Human

  • Animatable-human methods address body reconstruction from motion sequences and disentangling non-rigid deformation through canonical fields, blending, or motion regularization.A-NeRF [66] conditions radiance fields on relative pose coordinates; AnimatableNeRF [55] uses a canonical radiance field and ray blending, while later works [62] [11] add complementary blending or cycle consistency.Source: §2.3 · p. 4

3. DNA-Rendering

DNA-Rendering combines a multiview capture system and diverse performance collection with annotations for rendering and animation research. Its annotation pipeline includes calibration, matting, keypoint estimation, and SMPLX fitting, with reported mask and fitting quality measures.

3.1. Dataset Capture

  • The capture system combines 60 RGB cameras, eight Kinect depth cameras, and 16 lighting boards, with synchronized streams recorded at 15 fps.Twelve cameras capture up to 4096 × 3000 resolution; lighting is set to 5600K ± 300K and 4500 Lux/m to capture texture and material effects.Source: §3.1 · p. 4
  • Each actor wears three outfits and performs at least three actions per outfit, with 15-second free-style sequences and additional A-pose and empty-background frames.Extrinsic calibration data are collected daily; color and intrinsic calibration data are collected when the system is adjusted.Source: §3.1 · p. 5

3.2. Dataset Statistics

  • The dataset combines 153 professional actors in special scenes with 347 normal performers in daily-life scenes, covering 153 special and 269 common subcategories.The design considers gender, age, skin color, actions, clothing, and makeup.Source: §3.2 · p. 5

3.3. Data Annotation

  • The annotation pipeline covers camera calibration, color calibration, image matting, and parametric-model fitting, with its stages illustrated in the pipeline figure.Camera calibration includes intrinsic and extrinsic parameters; the figure also identifies masks and keypoints as annotation components.Source: §3.3 · p. 5, Figure 2 · p. 5
  • Mask refinement with HSV filtering and GrabCut reduces the average error rate of problematic cases from 11% to 2%.The pipeline first uses a background matting model, then refines artifacts such as holes and noisy patches.Source: §3.3 · p. 6, Figure 3 · p. 6
  • The annotation pipeline's mean and median MPJPE are 30.20 mm and 29.80 mm, respectively, compared with an oracle fitting accuracy of 29.34 mm in Human3.6M.SMPLX parameters are estimated after 2D keypoint detection and multiview 3D keypoint triangulation.Source: §3.3 · p. 6

4. Benchmarking Human-centric Rendering

The benchmark evaluates human-centric rendering across novel views, poses, and identities, using difficulty splits to probe performance. Its findings expose challenges in texture, deformation, interaction, and cross-dataset generalization.

4.1. Data Splits

  • Splits probe clothing deformation, texture, motion difficulty, and interaction size; testing uses 13 splits with three sequences each.The overall training split contains 400 sequences distributed across human factors and difficulty levels.Source: §4.1 · pp. 6–7, §4.3 · p. 7

4.2. Task Definition

  • Case-specific methods are tested on unseen views and held-out sequence frames; generalizable methods train on 400 sequences and render unseen identities.Novel view synthesis uses 42 training views and 18 unseen camera poses; novel pose animation trains on the first 80% of frames and tests on the last 20%.Source: §4.2 · p. 7

4.3. Benchmark Results

  • Rendering quality falls as split difficulty rises; among case-specific methods, A-NeRF leads PSNR, NeuralBody SSIM, and HumanNeRF LPIPS.NeuralBody and A-NeRF produce fewer background artifacts, while HumanNeRF better preserves high-frequency textures.Source: Figure 4 · p. 8, §4.3 · p. 8, Figure 5 · p. 9
  • As pose difficulty rises, quality drops; deformation and interaction remain hardest, with Peking-opera costume deformation failing across methods.NeuralBody and AnimatableNeRF cannot model interactive objects, while A-NeRF stretches objects under large poses.Source: §4.3 · pp. 8–9
  • Human-prior generalizable methods outperform category-agnostic methods with greater robustness on novel identities.Among category-agnostic methods, IBRNet outperforms PixelNeRF and VisionNeRF; the paper links stronger generalization to human priors and observed appearance references.Source: §4.3 · p. 9, Table 3 · p. 9

4.4. Cross-dataset Comparison

  • DNA-Rendering training benefits cross-dataset generalization, with richer data supporting more plausible results across domain gaps than ZJU-MoCap or HuMMan.HuMMan generalizes poorly to other datasets, including simple ZJU-MoCap cases, despite its large data volume.Source: §4.4 · p. 10, Figure 7 · p. 11

5. Conclusion

DNA-Rendering combines a diverse multiview human-performance repository with benchmarks and cross-dataset experiments for human-centric rendering. Its documentation details actor and clothing attributes, capture hardware, and data handling.

5. Conclusion

  • The project presents a large, high-fidelity human-centric rendering repository, benchmarks for three major tasks, and cross-dataset experiments on generalization capability.The three tasks are novel view synthesis, novel pose animation, and novel identity rendering.Source: §5 · p. 11

Appendix

  • The appendix covers dataset statistics, hardware and collection protocols, annotation and toolchain comparisons, benchmark settings and results, competing datasets, and future work.Source: Appendix · p. 15

A.1. Dataset Statistics

  • The dataset includes 500 actors, 269 action definitions, and 153 professional actors; its clothing repository has 527 items across 50 types, with three outfits per performer.Actors are distributed by ethnicity at a 4:3:2:1 ratio for Asian, Caucasian, Black, and Hispanic individuals.Source: §A.1 · p. 15

A.2. Meta Attributes

  • Action-sequence metadata records actor demographics, clothing colors and types, action content, and visual details of interactive objects.Actor records include name, gender, ethnicity, age, height, and weight.Source: §A.2 · p. 15

A.3. Hardware Construction

  • The camera array combines 48 industrial 5MP cameras, 12 12MP cameras, and eight depth cameras; depth data provide coarse geometry, while Kinect RGB is excluded for poor color consistency.The camera layout places 5MP cameras at three heights and 12MP cameras at two intermediate heights.Source: §A.3 · p. 16
  • Sixteen 5600K ± 300K flat lights provide scene illumination, while a triggered camera network has a maximum synchronization error below 2 ms across industrial cameras.The Kinect synchronization error is 1.12 ms in theory.Source: §A.3 · pp. 16–17

A.4. Data Collection Protocol

The collection protocol combines routine system checks with actor performances in varied outfits and actions, then verifies and processes the captured data.

A.4. Data Collection Protocol

  • Before collection, staff check camera settings, network status, synchronization, and extrinsic calibration; actors provide authorization and recorded appearance attributes.Four to six actors are invited per day, and performers rehearse before formal shooting.Source: §A.4 · p. 17
  • Normal actors perform at least three selected daily actions, while professional actors perform at least six skill- or costume-specific footage sets; each of two additional outfits receives three more actions.Source: §A.4 · p. 17
  • The protocol captures blank backgrounds and A-pose references, reviews multiview previews for invalid footage, and converts RAW images to BMP before video compression and upload.Invalid performances are repeated after review.Source: §A.4 · p. 17

A.5. Limitations on Data Collection

The appendix describes collection constraints and calibration and annotation refinements, reporting improved color and keypoint consistency and SMPLX fitting accuracy.

A.5. Limitations on Data Collection

  • Uniform lighting and 15 fps capture are current collection conditions; the authors identify variable lighting, multiple-person fields of view, high-speed capture, and multisensory data as future upgrades.Large movements beyond camera fields of view and transmission-related frame loss can also cause capture failures, which operators inspect and recapture.Source: §A.5 · p. 17

B.1. Camera Intrinsic Calibration

  • Intrinsic calibration uses a 3 × 3 image grid and rotates checkerboards in each region to cover the camera view and estimate distortion.The long-lens setup makes distortion estimation important when body parts approach image boundaries.Source: §B.1 · p. 18, Figure S4 · p. 18

B.2. Camera Color Calibration

  • ∆E00 consistency between two calibrated cameras improves from 37.79 to 4.15 after color correction.Calibration uses hardware adjustment followed by a polynomial mapping from sampled RGB values to standard color values.Source: Figure S5 · p. 18, §B.2 · p. 18

B.3. Keypoints

  • Below 30 pixels, the optimized 3D keypoints’ mean reprojection error falls after view selection, fixed main-limb bone lengths, and human-prior outlier filtering.Missing keypoints are interpolated after outlier removal.Source: §B.3 · pp. 18–19, Figure S6 · p. 19

B.4. Parametric Model

  • SMPLX fitting estimates shape from static A-pose bone lengths, then optimizes pose and translation for dynamic sequences; shape and pose optimization are decoupled for more stable results.Shape is held consistent across frames, while pose optimization uses keypoint and joint-angle prior energies.Source: §B.4 · p. 19
  • 15.87 mm is the MPJPE for hand-only keypoints; body-only keypoint MPJPE is reported as on par with Human3.6M optical motion capture.The authors evaluate body-only, hand-only, and all keypoints separately.Source: §B.4 · p. 19, Figure S7 · p. 19

B.5. Comparison to Other SMPLX Fitting Methods

The proposed SMPLX fitting pipeline leads the compared methods across 2D and 3D metrics with a favorable runtime trade-off. A hybrid matting refinement also reduces the reported overall error rate.

B.5. Comparison to Other SMPLX Fitting Methods

  • The fitting pipeline triangulates and filters 3D keypoints before SMPLX fitting, a two-step strategy the authors describe as faster and more robust to outliers than direct reprojection fitting.Its remaining drawback is dependence on the first-stage triangulation results.Source: §B.5 · pp. 19–20
  • The proposed SMPLX pipeline outperforms the compared fitting methods across body, hand, face, and whole-body 2D and 3D metrics, with a better performance-efficiency trade-off than alternatives.MultiviewSMPLify achieves second-best performance but takes an order of magnitude longer; the table reports fitting metrics and runtime.Source: §B.5 · p. 20, Table S1 · p. 20

B.6. Matting and Segmentation Refine

  • GraphCut refines CNN-predicted masks for corner cases including noisy backgrounds, broken bodies, and missing bodies or objects.The authors report that this classical method fits well with the CNN-based method on these cases.Source: §B.6 · p. 20
  • 11% to 2% is the overall matting error-rate change after hybrid refinement, compared with the CNN-based method alone.Three annotators graded 500 randomly selected multiview sequences, and error rates decreased across all corner-case categories.Source: §B.6 · p. 20

B.7. Quality Control of Auto Annotation Results

The benchmark compares human-rendering methods across novel-view settings, while the annotation workflow combines manual review with automatic processing to check SMPLX and matting quality.

B.7. Quality Control of Auto Annotation Results

  • SMPLX review uses multi-view overlays, binary filtering, five-level quality grading, and keypoint reannotation followed by rerunning the fit until valid models are obtained.Manual checks cover both SMPLX and matting outputs from the annotation pipeline.Source: §B.7 · p. 20
  • Matting annotations are graded A–C: A and B masks are acceptable, while C marks failures; benchmark train and test cases were manually selected for high annotation quality.B-level errors affect no more than one-third of the human-body area; the mask rating will be released as confidence.Source: §B.7 · pp. 20–21

C.1.1 Static Methods

  • Static baselines anchor novel-view performance on frozen frames; Instant-NGP was modified for cameras with different intrinsic parameters, while NeuS required no special adaptation.Instant-NGP uses multi-resolution hash embedding and a smaller network; NeuS combines a neural radiance field with a neural SDF.Source: §C.1.1 · p. 21

C.1.2 Dynamic Methods

  • Dynamic methods require dataset-specific adaptations, including four balanced views for NeuralVolumes, non-foreground pixel sampling for A-NeRF, and dense multiview training for NeuralBody and HumanNeRF.HumanNeRF models that collapse on some sequences are excluded from reported benchmark numbers.Source: §C.1.2 · pp. 21–22

C.1.3 Generalizable Methods

  • The novel-identity benchmark covers category-agnostic methods and methods with human structure priors; KeypointNeRF renders a square image before cropping to the requested resolution.KeypointNeRF uses 24 SMPL main-skeleton keypoints as priors.Source: §C.1.3 · p. 22

C.2.1 Novel View Synthesis

  • A-NeRF, NeuralBody, and HumanNeRF achieve the best numbers on most novel-view test splits; HumanNeRF has the best LPIPS and visual sharpness, while methods show distinct artifact trade-offs.NeuralVolumes ranks in the top three for PSNR but produces smoother results; NeuralBody and AnimatableNeRF have typically higher SSIM but struggle with large interactive objects.Source: §C.2.1 · pp. 22, 24, Figure S9 · p. 23

C.2. Benchmark Details

  • Dynamic novel-view evaluation uses 13 splits and 39 sequences, training per sequence on the first 80% of frames and testing every 45 frames from 18 unseen camera poses.Static methods are trained separately on each frame; evaluation uses half-resolution images.Source: §C.2.1 · p. 22

C.2.2 Novel Pose Animation

Novel-pose tests show complementary metric strengths among methods but expose difficulty with unseen local motion, deformation, and interactions. Novel-identity results favor human-prior methods, while VisionNeRF improves on PixelNeRF in the reported comparisons.

C.2.2 Novel Pose Animation

  • Novel-pose tests train on the first 80% of frames and evaluate pose sequences from the final 20% with a 15-frame skip, using the novel-view test views and resolution.Inputs are SMPL parameters or image features, depending on the method.Source: §C.2.2 · p. 24
  • A-NeRF achieves the best PSNR, NeuralBody the best SSIM, and HumanNeRF the best LPIPS on novel-pose animation; current methods still struggle with deformation and interactions.NeuralVolumes drops substantially, especially on motion splits, and its global warping can miss local motion or global scale in unseen poses.Source: §C.2.2 · pp. 24, 26, Figure S10 · p. 25

C.2.3 Novel Identity Rendering

  • Novel-identity evaluation trains on 400 sequences covering all categories and difficulties, then tests unseen identities on 18 views using four balanced source views.Category-agnostic methods receive segmentation and camera parameters; methods with human priors also receive fitted SMPLX or 3D keypoints.Source: §C.2.3 · p. 26
  • Human-prior methods generally render more precise shape and texture than category-agnostic methods on hard novel-identity cases, especially Texture-Hard and Deformation-Hard.VisionNeRF also achieves better average visual quality and higher scores than PixelNeRF in the reported comparisons.Source: §C.2.3 · p. 26, Figure S11 · p. 26

C.2.4 Benchmarks with Sparse View Training

Sparse-view training changes novel-view rankings and widens performance gaps for some methods, while sparse novel-pose results track dense-view trends more closely. The evaluation uses four balanced training views.

C.2.4 Benchmarks with Sparse View Training

  • Sparse-view models train on four balanced cameras; the benchmark frames sparse rendering as a step toward portable avatar generation from less structured data.The sparse and dense novel-view and novel-pose experiments otherwise use the same settings.Source: §C.2.4 · pp. 26–27
  • With sparse training, NeuralBody, AnimatableNeRF, and HumanNeRF rank in the top three across all novel-view metrics, unlike the dense-view results; HumanNeRF leads PSNR and LPIPS on most splits and NeuralBody leads SSIM.Sparse-view results show a larger performance drop than dense-view results, particularly as deformation and texture difficulty increase.Source: §C.2.4 · p. 27, Table S3 · p. 27, Figure S12 · p. 28
  • Sparse novel-pose results follow dense-view trends more closely, with HumanNeRF and NeuralBody strongest across the three metrics and a relatively smaller dense-to-sparse performance drop.The authors conclude that neither dense nor sparse observations suffice for benchmark methods to learn a compact dynamic field for unseen poses.Source: §C.2.4 · pp. 27–28, Table S3 · p. 27

D. Cross-dataset Details

  • The cross-dataset supplement describes dataset selection, implementation and settings, results, and an analysis of color consistency across multiview datasets.A cross-dataset affinity map reports PSNR, SSIM, and LPIPS* in separate matrices.Source: D. Cross-dataset Details · p. 28

D.1. Compared Datasets

The cross-dataset comparison selects human-centric datasets with foreground segmentation and dense views, then standardizes source views, image crops, training, and evaluation volumes to compare generalizable methods.

D.1. Compared Datasets

  • ZJU-MoCap, HuMMan, and GeneBody are selected for comparison; their documented trade-offs include limited diversity or interaction, image-quality and color-consistency issues, and differences in capture coverage.ZJU-MoCap has 10 sequences but limited clothing and motion diversity; HuMMan emphasizes muscle-related actions with marginal clothing diversity; GeneBody covers a wider range of clothing, motion, and interactions.Source: §D.1 · pp. 28–29

D.2. Detailed Settings

  • For fair dataset comparison, the authors evaluate only category-agnostic PixelNeRF [81], IBRNet [75], and VisionNeRF [38], cropping subjects from original images with square bounding boxes.Selected datasets require foreground segmentation annotations and dense camera views; Human3.6M has four RGB cameras, while CMU Panoptic and AIST++ lack official segmentation annotations.Source: §D.1 · p. 28, §D.2 · p. 29
  • All datasets use four balanced reference views, the same learning rate and 200k training iterations, and official splits with test volumes adjusted by dataset.Test-frame and view sampling differs across DNA-Rendering and GeneBody, ZJU-MoCap, and HuMMan.Source: §D.2 · p. 30

D.3. Additional Results

The supplementary results report stronger generalization with greater dataset volume and diversity, while color consistency is associated with steadier performance across views.

D.3. Additional Results

  • The appendix reports individual general-scene-method results and qualitative cross-dataset examples; cross-dataset absolute values are lower than novel-identity results under the larger foreground proportion.Figure S15 shows samples arranged by test dataset and training dataset.Source: §D.3 · p. 30, Figure S15 · p. 31

D.3.1 In-domain

  • HuMMan [7] and ZJU-MoCap [56] achieved the strongest in-domain results, but DNA-Rendering’s greater volume and diversity improved performance over GeneBody [13] across all three methods.The comparison notes that GeneBody [13] has about 10% of DNA-Rendering’s data volume.Source: §D.3.1 · p. 30

D.3.2 Cross-domain

  • DNA-Rendering-trained models achieved the best cross-domain results in most first-row tests; on DNA-Rendering splits, GeneBody [13] performed best, while HuMMan [7] and ZJU-MoCap [56] dropped on difficult cases.DNA-Rendering and GeneBody [13] also showed only marginal degradation when evaluated on other datasets.Source: §D.3.2 · pp. 30–31, Figure S14 · p. 30

D.4. Impact of Color Consistency

  • Color-inconsistent training datasets showed higher view-performance variance in-domain and larger performance drops and variance cross-domain; DNA-Rendering and GeneBody [13] showed small cross-view variations.The analysis compares average PSNR across views and selects test views with close angles to the nearest source view.Source: §D.4 · p. 31, Figure S16 · p. 32

E. Future Work

The authors propose tools and a leaderboard to align evaluation, while identifying unresolved matting refinement and additional possible benchmark uses.

E. Future Work

  • The dataset and benchmarks are available under a Creative Commons license with usage agreements; a web leaderboard and easy-to-run tools are planned to reduce evaluation divergence.The planned resources aim to better align standards across institutions.Source: E. Future Work · p. 32
  • Matting refinement remains imperfect: per-frame human checks are impractical, so released sequences exclude cases with major artifacts across views; 3D refinement attempts produced blurry edges.The authors plan to investigate more robust tools.Source: E. Future Work · p. 32
  • The three task benchmarks are a kickoff; potential further uses include garment modeling and animation, and human shape completion.The authors invite the community to develop additional downstream tasks.Source: E. Future Work · p. 32
Loading 2307.10173v2…