Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer

• Association locates evidence on 3D support, whereas reconciliation determines identity, semantic agreement, and granularity; therefore correct association alone does not establish identity.The framework separates these decisions because projection, rendering, attention, and temporary reconstruction can implement association differently.Source: §1 · pp. 2–3
• LAF represents transfer with Generate, Associate, Reconcile, Fuse, and Persist/Query operators that form an auditable decision trace.The operators describe decisions rather than mandatory software modules and may be jointly implemented or revisited through feedback.Source: §1 · p. 2
• The persistent-carrier contract records spatial support, semantic and identity state, uncertainty, provenance, and supported operations, defining what future queries can recover or corrections can revise.It identifies the first stage after which discarded hypotheses become unrecoverable without rerunning an upstream model or rebuilding the scene.Source: §1 · pp. 2–3
• LAF was operationalized as a representation-neutral audit of 161 systems available through 7 August 2026 across six carrier families, with no additional analytical stage required after final confirmation.The corpus stresses calibrated, renderer-mediated, learned, temporary, online, relational, dynamic, and feed-forward mechanisms.Source: §1 · p. 3
• The audit finds that carrier design fixes query and correction boundaries, while rendered-view, native-3D, and proposal-level evaluations answer different questions.Systems with similar masks or reconstructions can retain different evidence and fail under different perturbations.Source: §1 · pp. 2–3
• Efficiency and vocabulary qualifiers such as training-free, real-time, open-vocabulary, and generalizable are meaningful only when scoped to a stage and complete cost ledger.This finding motivates reporting rules and design requirements for revisable 3D perception.Source: §1 · p. 3

2.1 Operator model and decision trace

• LAF represents each system as Generate, Associate, Reconcile, Fuse, and Persist/Query, recording supervision, discarded state, and costs by implemented decision rather than module naming.The operators expose image evidence, assign 3D support, establish hypotheses, materialize reusable state, and read or change that state.Source: §2.1 · p. 4

2.2 Persistent carrier contract

• The persistent carrier contract specifies spatial support S, semantic state Z, identity/topology I, uncertainty U, provenance P, and supported operations O, separating retained state from query-time services.Different representations are comparable only when they expose the same contract under explicit conversion, and correction costs follow the retained tuple.Source: §2.2 · p. 4

2.3 First irreversible loss

• The first irreversible loss is the earliest query-relative stage where a needed hypothesis is absent from retained state and cannot be reconstructed from later inputs.False merges, lossy fusion, discarded provenance, and narrow query interfaces can respectively remove identity, alternatives, explanations, or operations.Source: §2.3 · pp. 4–5, Figure 2 · p. 5

2.4 Framework construction corpus

• The validation corpus contains 166 indexed methods: 161 satisfying the framework contract, four boundary or adjacent cases, and one excluded after screening, from 165 parsed PDFs totaling 2,277 pages.Eligible systems transfer broadly pretrained 2D knowledge into material 3D segmentation or persistent 3D labels and include substantive association, reconciliation, fusion, or persistence decisions.Source: §2.4 · p. 5

2.5 Audit procedure

• Each method receives a five-pass trace covering generated evidence, support, identity and conflict decisions, fusion and discarded alternatives, and carrier operations, with claims checked against primary-source locations.The audit does not pool accuracy across incompatible geometry, vocabulary, proposal, prompt, output, or denominator protocols.Source: §2.5 · p. 6

2.6 Framework stress tests and observed coverage

• Stress tests across representations, mechanisms, state evolution, and boundary cases found no required sixth decision stage or missing canonical mechanism at the 2026-08-07 cutoff.This supports analytical saturation rather than proving that future work cannot extend the framework.Source: §2.6 · p. 6
• Cross-representation traces show that similar field or Gaussian categories can differ in identity timing, update paths, and correction boundaries, as illustrated by OpenScene [Peng et al., 2023], SA3D [Cen et al., 2023], Gaussian Grouping, and OGScene3D [Zhu et al., 2026].LAF distinguishes their association, reconciliation, fusion, persistence, and feedback choices rather than grouping them by representation alone.Source: §2.6 · p. 6

2.7 Validity and limitations

• LAF is an analytical instrument whose validity depends on faithful reconstruction from published evidence; omissions, qualitative runtime costs, and the absence of prospective double-coding remain limitations.The framework focuses on systems with a material 3D segmentation pathway, while other tasks enter only as boundary tests.Source: §2.7 · pp. 6–7

3 Operational Definitions and Correspondence Operators

• The framework distinguishes evidence, support, carrier state, and query operations because different transferred objects permit different valid operations.Evidence may include colors, masks, language embeddings, features, identities, affinities, or uncertainty; carriers persist the state available after construction.Source: §3 · p. 7, §3.1 · p. 7

3.1 Images, geometry, and evidence

• Association weights pixel support for 3D elements, while reconciliation resolves identities or scales, fusion creates persistent state, and querying maps that state to outputs.The notation explicitly separates physical correspondence a from semantic state z: correct projection does not imply shared object identity or label.Source: §3.1 · p. 7

3.2 Calibrated projection and visibility

• Calibrated lifting requires positive depth, image bounds, and visibility checks, with hard or confidence-weighted support depending on residuals, view angle, visibility, or confidence.Treating occluded points as visible creates false supervision, whereas overly strict tolerances remove valid evidence; OpenScene, ConceptFusion, SAM3D, and Open3DIS use calibrated association under different assumptions [Peng et al., 2023, Jatavallabhula et al., 2023, Yang et al., 2023, Nguyen et al., 2024].Source: §3.2 · p. 7

3.3 NeRF volume rendering as soft association

• NeRF volume-rendering weights provide differentiable, partially visible association for transferring supervision into feature or mask fields, but depend on learned density accuracy.LERF stores scale-conditioned language relevance, SA3D optimizes a prompted mask field, and OpenNeRF distills localized OpenSeg features [Kerr et al., 2023, Cen et al., 2023, Engelmann et al., 2024].Source: §3.3 · p. 7

3.4 Gaussian alpha compositing

• Gaussian lifting uses non-equivalent policies including ray intersection, footprint overlap, maximum contribution, and marginal contribution, while semantic storage may be attached to primitives or external fields.Gaussian Grouping, LangSplat, FlashSplat, Dr. Splat, InfoGS, and FMGS exemplify these different choices [Ye et al., 2024, Qin et al., 2024, Shen et al., 2024, Kim et al., 2025, Zhang et al., 2025e, Zuo et al., 2025].Source: §3.4 · p. 8

3.5 Identity, semantics, and granularity

• Identity, semantics, and granularity are separate variables: carriers may preserve class-agnostic identities, language fields, hierarchies, or affinity graphs with different revision and memory tradeoffs.Hard labels discard alternative memberships, whereas affinity graphs preserve more options but require memory and clustering.Source: §3.5 · p. 8

3.6 Training and vocabulary regimes

• Per-scene, cross-scene, and online regimes shift costs between construction and deployment, while open-vocabulary status depends on whether stored representations answer new text queries without rerunning upstream 2D models.Vocabulary may be fixed before training, encoded during construction, or attached after geometry is frozen; query latency alone cannot rank these regimes.Source: §3.6 · p. 8

3.7 Terminology crosswalk

• The terminology crosswalk preserves distinctions among neighboring terms because they can differ in mechanism, supervision, evaluation, or deployment cost.Table 2 records aliases as operational trace definitions rather than exact synonyms, favoring the most specific supported term.Source: §3.7 · p. 8, Table 2 · p. 9

4 Generate: What Comes from 2D?

• Generation fixes the pipeline’s information ceiling: 3D reconciliation cannot recover categories, parts, or instances that source models never expose.The decisive inputs include object granularity, identity, confidence, and when vocabulary enters the system.Source: §4 · p. 9

4.1 Masks provide support but not 3D identity

• Mask generators provide object support but do not establish 3D identity, while proposal topology imposes provisional scale and coverage limits.Systems place identity decisions in independent masks, 3D prompting, point-space resolution, tracking, or learned fields, and may retain whole, part, and subpart evidence.Source: §4.1 · p. 9

4.2 Language can name pixels, regions, or entities

• Language enters as pixel features, proposal embeddings, boxes, text-conditioned masks, or generated descriptions, whose spatial support and provenance affect supervision reliability.Generated text can add attributes and affordances but may hallucinate entities, omit small objects, or repeat correlated language biases.Source: §4.2 · p. 10

4.3 Self-supervised features contribute structure

• Self-supervised features add structural evidence, but their presence does not determine the output vocabulary or evaluation task.Multi-teacher systems must allocate competence and preserve source information or calibration, because incompatible errors can otherwise become a confident fused vector.Source: §4.3 · p. 10

4.4 Tracking, depth, and pose generate relational evidence

• Tracking, depth, and pose generate relational evidence that can bridge views but also propagate identity switches, correspondence errors, and proposal-topology mistakes.Measured or inferred geometry determines where evidence is lifted; cached or tracked errors may recur across later prompts and queries.Source: §4.4 · pp. 10–11

4.5 Generation should expose uncertainty and absence

• Generation should preserve confidence, provenance, competing evidence, tracker reliability, vocabulary source, and explicit absence because association and fusion cannot reconstruct collapsed information.Cross-view variance, visibility-aware reliability, and retained confidence support later observation requests or map revision.Source: §4.5 · p. 11

4.6 Generation can be offline, synthetic, or task-driven

• Generation may be online, offline, synthetic, action-driven, or non-RGB, so systems must report what the foundation model generates, what it only names, and which data or rendering assumptions persist.Offline corpora, canonicalization, geometric render channels, linguistic decompositions, and supervised adaptation can encode deployment-time biases even when inference is fast or masks are absent.Source: §4.6 · p. 11

5 Associate: From Image Evidence to 3D Support

• Association is a substantive operator: its visibility, weighting, and intermediary choices determine allowed 3D support, uncertainty, and downstream failure correlations, while identity still requires reconciliation.Figure 3 distinguishes hard projection, neural-field, Gaussian-renderer, and learned or temporary realizations; all locate candidate support without establishing cross-view physical identity.Source: §5 · p. 12, Figure 3 · p. 12

5.1 Calibrated lifting is hard correspondence with an explicit visibility policy

• Calibrated lifting uses projection plus visibility tests, but occlusion thresholds, view selection, and primitive choice make nominally projection-based systems incomparable.OpenScene and ConceptFusion transfer features to points, while SAM3D, OV-SAM3D, and Open3DIS transfer masks to points or proposals; Details Matter restricts overlap to co-visible superpoints [Peng et al., 2023, Jatavallabhula et al., 2023, Yang et al., 2023, Tai et al., 2024, Nguyen et al., 2024, Takmaz et al., 2023, Jung et al., 2025].Source: §5.1 · p. 12

5.2.1 Neural fields and Gaussian renderers

• Neural fields and Gaussian renderers replace hard correspondence with weighted contributions, but geometric errors can spread correct labels to wrong support and Gaussian lifting may target attributes, correlations, or votes.View admission and cross-view search also determine which evidence accumulates, as shown by OpenNeRF, Fun3DU, Wheat3DGS, and related systems [Kerr et al., 2023, Liu et al., 2023a, Engelmann et al., 2024, Cen et al., 2023, Ying et al., 2024, Kim et al., 2024, Corsetti et al., 2025, Zhang et al., 2025b].Source: §5.2.1 · p. 13

5.2.2 View selection and mask graphs

• View-selection and mask-graph methods define association through repeated co-visibility or shared third-view support, while PS3 fixes the 3D part inventory by choosing a 2D granularity ceiling [Yin et al., 2024, Yan et al., 2024, Yen et al., 2026].SAI3D links geometric primitives sharing masks, whereas MaskClustering links masks receiving common support from other views.Source: §5.2.2 · p. 13

5.2.3 Hybrid support operators

• Hybrid operators expose that “using 2D features in 3D” is underspecified: support may be point-wise, reconstructed, Gaussian-dominant, matrix-valued, or closed-form, with different costs and correction capacity [Lee et al., 2025a, Abou Zeid et al., 2026, Guo et al., 2025, Xue et al., 2025, Chacko et al., 2025, Shen et al., 2025, Marrie et al., 2025].LUDVIG attaches arbitrary DINO, SAM, or CLIP evidence without feature optimization, while optional graph diffusion belongs to reconciliation and must be costed separately.Source: §5.2.3 · p. 14

5.3.1 Temporary geometric intermediaries

• Temporary intermediaries can separate the representation used for association from the returned carrier, but the first incorrect assignment remains an irreversible error that reconciliation cannot reliably repair.PointGS uses 3DGS as a temporary visibility engine, while GeoSAM2 and MV3DIS use geometry to guide association; reporting only the final carrier hides where support errors enter [Song et al., 2026, Gong et al., 2026, Zhang et al., 2025d, Pan et al., 2025, Deng et al., 2026a, Zhao et al., 2026].Source: §5.3.1 · p. 14

5.3.2 Online and object-conditioned association targets

• Object-conditioned systems selectively acquire evidence or propose candidate parts before depth lifting, so parent errors become association errors; trajectory-based association makes track switches contaminate downstream masks, captions, and Gaussian features [Deng et al., 2026b, Zhang et al., 2025a, Rotondi et al., 2025, Kim and Kim, 2026, Fu et al., 2026, Tan et al., 2026].ProFuse reuses one dense correspondence graph for Gaussian geometry and cross-view mask identity, so a bad match can seed the wrong surface and merge the wrong semantic evidence.Source: §5.3.2 · p. 15

5.3.3 Association shared with reconstruction

• When association shares reconstruction, retrieval, keypoint, or tracking correspondence changes which conflicts reach the carrier but does not remove the need to expose discarded masks, merge provenance, and full construction costs [Dou et al., 2024, Hu et al., 2025a, Tang et al., 2024, Zhao et al., 2025].CoSegGaussians associates before compression, CCGS revises mask overlap with pointmaps, and 3DIML repairs omissions only after provisional identities are formed.Source: §5.3.3 · p. 15

6 Reconcile: Identity, Conflict, and Granularity

• Reconciliation must preserve geometry, identity, and semantic scale because association alone cannot determine shared entities, conflict resolution, or valid part–whole relations.A chair seat and chair may be split at object scale but hierarchical at part scale, while adjacent chairs with similar language features must remain distinct.Source: §6 · p. 15

6.1 Identity has an establishment time

• Identity establishment determines the correction boundary: externally tracked, geometry-first, and learned carriers differ in when identities become fixed, but later semantic processing cannot repair missed, merged, or wrongly tracked objects.Learned codes and relational carriers can represent richer structure, yet still depend on projected masks or association priors; irreversible proposal pruning can remove small objects before classification.Source: §6.1 · p. 16

6.2 Semantic conflict is not identical to identity conflict

• Semantic disagreement is distinct from identity disagreement: contextual or language-driven reconciliation can resolve conflicts, but unsupported labels, semantic-code collapse, and geometry-changing corrections require abstention, provenance, and joint appearance–segmentation audits.False entity masks can corrupt contrastive alignment, while feedback methods may split, prune, or alter geometry; competing views or graph edges should not be collapsed into one confident answer.Source: §6.2 · p. 17

6.3.1 Scale and hierarchy as query variables

• Granularity is a query variable rather than a universally recoverable partition: continuous scales and hierarchies cannot recover absent 2D parts, so overlapping memberships, provenance, confidence, and part–whole structure can defer destructive choices.PS3 improves medium-scale identity but remains weak on fine-granularity coverage, with MultiScan gains concentrated at AP50 and missed tiny hooks on ScanNet++; hierarchy interfaces are therefore not interchangeable.Source: §6.3.1 · pp. 17–18

6.4 Reconciliation extends to online, relational, and dynamic carriers

• Online, relational, and dynamic carriers reconcile node identity, relation identity, teacher agreement, and temporal membership separately, so evaluation must test lifecycle events, occlusion, and topology change rather than only final-scene accuracy.Different systems use greedy assignment, Hungarian matching, graph merging, adaptive affiliation fields, recurrent memory, or repeated re-inference, exposing distinct assumptions about fragmentation, collision, disappearance, and reappearance.Source: §6.4 · pp. 18–19

7.2 Learned fields trade query efficiency for optimization and compression choices

• Learned fields and Gaussian carriers trade query efficiency against optimization, compression, and placement choices that constrain representable overlaps, relations, and query operations.Neural fields provide smooth renderer-consistent queries but can entangle geometry and semantics; Gaussian systems distribute semantics across latents, quantized features, external indices, or decoder correlations.Source: §7.2 · p. 20

7.3 Discrete fusion preserves topology but can hide uncertainty

• Discrete fusion yields directly usable partitions and edits, but hard assignments hide alternative correspondences and can confuse improved rendered masks with complete native-3D labels.Overlap merging, contribution voting, local majority filtering, and image-space refinement differ in whether they alter the persistent carrier or only the displayed result.Source: §7.3 · p. 21

7 Fuse: Combining Evidence and Constructing the Carrier

• Fusion systems relocate costs and assumptions rather than eliminating them, including training, pseudo-label generation, proposal coverage, thresholds, and workload-dependent prompt efficiency.Dataset-trained and feed-forward carriers avoid per-scene optimization but shift expense into data generation and training; known-prompt detectors can lose advantages under many sequential new queries.Source: §7.3 · p. 21, §7.4 · p. 21

7.4 Fusion should be evaluated as a full ledger

• Fusion must be evaluated across reconstruction, preprocessing, association, optimization, storage, and query costs because last-operation latency can conceal substantial upstream work.LEGaussians, RegionPLC, and Octree-Graph illustrate how reported efficiency depends on dense extraction, training, captioning, projection, and merging costs.Source: §7.4 · p. 21

7.1 Averaging is a model, not a neutral default

• Averaging is a modeling choice: means and majority clusters simplify updates but can collapse contradictory evidence, so carriers should retain uncertainty or provenance.Confidence-weighted temporal averaging, robust aggregation, prototype consensus, and learned smoothing each impose assumptions about reliability, disagreement, or representative features.Source: §7.1 · pp. 19–20, §7.5 · p. 22

7.5 Compact and temporal carriers make information loss explicit

• Compact and temporal carriers expose irreversible information loss: filtering, Top-K selection, pruning, and early online merges can remove evidence needed for later queries or corrections.Mean updates and dominant-feature clustering hide multimodality, while sparse mixtures, temporal fields, and persistent instances preserve different amounts of disagreement and revisability.Source: §7.5 · p. 22, §7.4 · p. 22

7.6 Fusion can preserve relational structure

• Fusion can preserve relations rather than descriptors, using semantic-spatial graphs or multimodal affinities to support relational queries while inheriting missed proposals, incorrect edges, and cue-specific biases.GeoCGA aligns semantic-spatial and object graphs, while CMAT and SAP preserve pairwise or heterogeneous geometric-semantic organization instead of forcing exact feature equality.Source: §7.4 · p. 21, §7.6 · p. 22

8 Persist, Render, and Query

• Persistent carriers differ in stored geometry, semantics, identity, uncertainty, provenance, query operations, and correction boundaries despite producing visually similar colored segmentations.Table 3 compares carrier families by natural output, main advantage, and correction boundary, showing that visualization does not define a shared query contract.Source: §8 · p. 22, Table 3 · p. 23

8.1 Persistent carriers expose different query surfaces

• Point, field, Gaussian, object, graph, and memory carriers expose different query surfaces, from text lookup and rendering to planning, temporal interaction, and revisable multi-field reasoning.Carrier design determines whether systems persist scene-specific features, discrete identities, transferable parameters, relations, interaction caches, or raw evidence; this also determines error scope and correction operations.Source: §8.1 · pp. 23–24

8.2 Open vocabulary has a timing

• “Open-vocabulary” depends on when language binds: before training, after construction, during proposal selection, or through query-time reasoning and optimization.Later binding improves flexibility but raises query cost and cannot recover missing persistent proposals; open inference text also does not guarantee open perceptual coverage.Source: §8.2 · pp. 24–25

8.3 Rendering and native-3D extraction are different evaluation domains

• Rendered relevance maps, native-3D masks, query latency, storage, update consistency, and graph accuracy evaluate different promised interfaces and should not be ranked together without conversion protocols.Novel-view localization, native topology, repeated-query efficiency, online updates, and edge, retrieval, or navigation performance each test distinct carrier capabilities.Source: §8.3 · p. 25

8.4 Hierarchies and graphs expose structured query interfaces

• Hierarchies and graphs make returned support and mutable state explicit, enabling object updates, parent-aware retrieval, planning, provenance-rich memory, and structured correction beyond unary semantic masks.Persistent raw images, symbolic edges, probabilistic graphs, compressed attributes, vector indexes, and voxel-group captions impose different retrieval and stale-evidence boundaries.Source: §8.4 · p. 25

8.5 Query-time and dynamic carriers change the reuse contract

• Query-time and dynamic carriers trade persistent reuse against recomputation, memory, temporal state, and update risk, so benchmarks must report identity transitions, recovery, and interface-specific metrics.Streaming query banks can make mistaken merges affect future evidence, while zero-shot, temporal, and click-conditioned systems expose different amortization and reuse contracts.Source: §8.5 · p. 26

Protocol comparability

• Metric comparisons are valid only when prediction object, vocabulary, source geometry, and query protocol match across systems.The protocol separates rendered-pixel, native-point, explicit-Gaussian, oracle-mask, and proposal-based evaluations instead of combining them into one leaderboard.Source: §9 · p. 26

System-specific protocols and costs

• IQGS reports rendered-view class-agnostic instance mIoU and a no-finetuning scene-transfer test, whereas OV3DSeg-VGGT evaluates semantic probabilities on predicted feed-forward geometry [Gao et al., 2026, Zhou et al., 2026].Neither denominator is interchangeable with open-vocabulary AP on fixed ScanNet geometry; IQGS reports 30k A100 iterations without normalized wall time, while numerical cost and accuracy claims are withheld for OV3DSeg-VGGT because only publisher HTML was locally verifiable.Source: §9 · p. 26

Output-contract distinctions

• Rendered-view, native-3D, proposal-level, and downstream evaluations are not interchangeable because visible masks can conceal differences in 3D support, topology, matching, query, and action policy.Cross-method comparison requires controlling geometry, views, prompts, conversions, proposal sources, and oracle access.Source: Figure 4 · p. 27, §9 · p. 27

9.1 The evaluation object comes before the metric

• Rendered-view and native-3D scores are separate protocol families because each depends on different prediction objects, visibility assumptions, and conversion algorithms.Rendered scores can remain high despite incomplete occluded topology; converting between rendered and native carriers requires sampling, thresholding, visibility, or rasterization.Source: §9.1 · p. 27, Table 4 · p. 28
• A shared dataset does not guarantee comparable supervision, outputs, or timing boundaries across rendered-mask, native-instance, decoder, and online-latency evaluations.DesktopObjects-360 demonstrates that one benchmark can support multiple evaluation domains and distinct protocol purposes.Source: §9.1 · pp. 27–28
• Instance-segmentation AP must identify proposal origin and oracle-mask access, because oracle AP isolates semantic reasoning and is not comparable with end-to-end AP.Protocols should also state whether one or several class predictions are allowed per instance.Source: §9.1 · p. 28
• Recent studies span rendered neural-field metrics, proposal AP, point mIoU, object-part IoU, and camera- or LiDAR-prompt segmentation, so one ranking would erase key regime differences.The omitted differences include prediction object, vocabulary, supervision, and sensor regime.Source: §9.1 · p. 28

Vocabulary and evaluation

• Base/novel results must report their class partition because changing B15/N4 to B10/N9 changes supervision and semantic difficulty; foreground-only metrics are not closed-set mIoU.PLA, RegionPLC, and PGOV3D report harmonic IoU alongside base and novel mIoU across partitions [Ding et al., 2023, Yang et al., 2024, Zhang et al., 2025d].Source: §9.2 · p. 28

Supervision and training

• Annotation-free means no manual target labels, not no training: RegionPLC and PGOV3D use generated supervision and training, while MPEC curates entity masks and descriptions [Yang et al., 2024, Zhang et al., 2025d, Wang et al., 2025b].These approaches measure reusable learned representations, unlike per-scene language fields that optimize a carrier for the test scene.Source: §9.2 · pp. 28–29
• Reusable learned representations and per-scene language fields are both valid, but they amortize construction cost and prior knowledge differently [Yang et al., 2024, Zhang et al., 2025d, Wang et al., 2025b].The distinction concerns whether learning is performed across scenes or optimized for the test scene.Source: §9.2 · p. 29

Language evaluation

• Free-form instruction evaluation requires dedicated prompts and correctness criteria, because fixed-category success does not establish compositional grounding.SOLE evaluates category, attribute, and affordance descriptions [Lee et al., 2025b], while OV3D-CG classifies proposals using an MLLM and contextual views.Source: §9.2 · p. 29

Stage-by-stage accounting

• A complete cost ledger must include reconstruction, 2D preprocessing, association, carrier construction or training, storage, and per-query inference.Omitting any stage changes the apparent conclusion; Table 5 records where each cost is paid and which omissions commonly make pipelines appear cheaper.Source: §9.3 · p. 29

Query workload

• Query reports should separate first-query latency, subsequent-query latency, and the number of prompts or classes processed together.Cached carriers can reduce marginal latency for repeated prompts, whereas dense language fields may be wasteful for a single known category list.Source: §9.3 · pp. 29–30

Training amortization

• Training and deployment costs must be reported together with pretraining data, hardware, geometry source, and excluded preprocessing.HumanCrafter combines roughly 6.2-second inference with seven days of training on eight A800 GPUs, while InfoGS reports a 16.3-minute shaping stage that presupposes reconstructed 3DGS and multiple foundation-model calls.Source: §9.3 · p. 30

Latency boundaries

• Latency figures are not contradictory when their measured stages are named, such as WildSeg3D’s 5–20 ms interaction after roughly 30 seconds of setup or Rethinking OVRF’s fast rendering after about 45 minutes of fitting and transfer.Any3DIS also reports hundreds of seconds for proposal generation, while DITR and SAM4D shift further cost into multi-GPU training.Source: §9.3 · p. 30

Evaluation domains

• Rendered-view, direct-asset, scene-graph, and relationship-guided evaluations expose different information and therefore cannot share one leaderboard.Lifting by Gaussians evaluates rendered views, Trace3D reports direct extraction and novel-view masks, and RelationField adds scene-graph recall and relationship-guided instance IoU.Source: §9.3 · p. 30

Comparability limits

• Comparisons must flag altered geometry, missing denominators, and unknown costs rather than infer efficiency from representation choice or labels such as “learning-free.”Semantics-guided reconstruction changes the carrier and appearance; NG-GS lacks complete end-to-end runtime, while LUDVIG still requires pretrained geometry, feature extraction, and inference-time graph diffusion.Source: §9.3 · p. 30

Discrete Gaussian interfaces

• SAGOnline’s 1.47-second mask aggregation and 27 ms rendering, and CDSeg’s roughly 2.23–3.74-second processing, are stage-level measurements rather than reconstruction-inclusive runtimes.SAGOnline assumes reconstructed 3DGS and a video foundation-model pass; CDSeg excludes external mask inference and prior reconstruction.Source: §9.3 · p. 30

Feed-forward systems

• Feed-forward speed must retain its amortization scope: PanSt3R takes about 2.3 minutes directly versus about 35 minutes with LUDVIG output, while PartField’s roughly ten-second clustered pass follows two weeks of training.CADRNet reports 372.7 GFLOPs per query, including 332.1 GFLOPs from frozen CLIP/Vicuna components; these outputs cannot form one speed ranking.Source: §9.3 · p. 30

9.4 Minimum reporting checklist

• Every quantitative row should identify dataset, protocol, output domain, geometry and proposal sources, supervision, full runtime boundary, and transient versus persistent memory.Claims such as “real-time,” “training-free,” “annotation-free,” and “generalizable” require this ledger rather than functioning as self-explanatory labels.Source: §9.4 · p. 31

9.4.1 Part, prompt, and agent workloads

• Part, prompt, and agent workloads require separate ledger rows because reported speed can exclude pseudo-labeling, adaptation, rendering, foundation-model calls, reconstruction, or reasoning.REALM reports 8.68 seconds per query, while its 354.72 FPS measures rendering; OpenSplat3D separates 20–45 minutes of fitting, clustering, and language aggregation.Source: §9.4.1 · p. 31

9.4.2 Online maps, hierarchies, and graphs

• Online maps, hierarchies, and graphs cannot share one efficiency column because metrics and costs span different carriers, stages, datasets, observation budgets, and upstream foundation-model calls.Open-Fusion reports 50 FPS for geometry versus 4.5 FPS with semantics, while OpenVoxel reports about three minutes for map construction and below one second per query under staged, non-reconstruction-inclusive timing.Source: §9.4.2 · pp. 31–32

9.4.3 Compact and dynamic carriers

• Compact and dynamic carriers require explicit temporal and reconstruction denominators because partial carrier timings and heterogeneous dynamic benchmarks do not support universal speed rankings.LightSplat reports roughly four seconds for distillation and 0.002 seconds per text query, whereas dynamic systems vary in annotated intervals, target-ID access, training regimes, and whether reconstruction is timed.Source: §9.4.3 · pp. 32–33

9.4.4 Specialized endpoints remain separate protocols

• Specialized endpoints remain separate protocols because identical metric names can refer to renderer outputs, native point instances, online masks, referring masks, mesh parts, or downstream edits.These systems also differ in sensors, teachers, language expansion, and prediction targets, so a common metric name cannot erase their distinct evaluation objects.Source: §9.4.4 · p. 33

10 Failure Propagation and Framework Diagnostics

• LAF attributes downstream errors to the first irreversible loss across Generate, Associate, Reconcile, Fuse, or Persist/Query, rather than the final classifier.Figure 5 and Table 6 organize propagation paths, causal tests, and recovery operations around this system- and query-dependent boundary.Source: §10 · pp. 33–34, Figure 5 · p. 34

10.1 Geometry and visibility errors corrupt support

• Geometry and visibility errors corrupt semantic support, so carriers should retain alignment confidence or geometric uncertainty instead of treating final scores as independent evidence.Strict or loose depth tolerances, floaters, blurred density, splat leakage, and predicted-map errors can all misassign semantics; uncertainty-driven re-observation may repeat the same artifact.Source: §10.1 · p. 34

10.2 Proposal topology imposes a hard ceiling

• Proposal topology imposes a hard ceiling: classifiers cannot split merged masks or recover omitted objects, while repairs trade fragmentation against erroneous merging or deletion.Semantic proposal generation relaxes class-agnostic limits but can bias topology toward captioned or supervised concepts.Source: §10.2 · pp. 34–35

10.3 Identity failures become persistent state transitions

• Identity errors become persistent state transitions, allowing tracker switches, early merges, and correlated consensus bias to contaminate masks, relations, retrieval, and navigation.Graph carriers therefore need merge provenance, edge confidence, local repair, and contradiction retention rather than only a winning identity.Source: §10.3 · pp. 35–36

10.4 Semantic disagreement can be averaged into confidence

• Semantic disagreement can be averaged into confidence when incompatible observations, generated captions, or systematic teacher bias collapse into plausible embeddings or training pairs.Source-specific confidence, provenance, negative evidence, and pseudo-label revision are needed because multi-view agreement helps only when errors vary across views.Source: §10.4 · p. 36

10.5 Granularity and language create valid but incompatible answers

• Granularity and language permit valid but incompatible answers across parts, objects, groups, and contextual interpretations, so open-vocabulary access does not guarantee semantically valid scale transitions.Systems should support abstention, expose supporting geometry and views, and test counterfactual context because persisted choices determine later cascades.Source: §10.5 · pp. 36–37

10.6 Hierarchies and graphs amplify upstream errors

• Hierarchies, object graphs, and functional graphs amplify upstream omissions or wrong assignments across partitions, relations, plans, and constrained queries.Diagnostics should perturb scale, thresholds, parentage, association, and observation order while requiring local rollback and testing missing ontology or evidence anchors.Source: §10.6 · p. 37

10.7 Dynamic, compressed, and generative carriers require recovery tests

• Dynamic, compressed, and generative carriers require recovery tests because discarded evidence, stale updates, wrong topology, and incorrect ontologies can remain coherent while irreparably limiting queries.Recommended tests include invalidation, revision, reindexing, reordered views, rare-query recall, contradictory evidence, track perturbations, and held-out paraphrases.Source: §10.7 · pp. 37–38

11 From Passive Segmentation to Agentic 3D Perception

• Agentic perception requires persistent identities, belief revision, active observation, and relational action support rather than answering one static-scene segmentation query.The transition is represented as a closed perception–memory–action loop in which memory updates fuse new evidence and persist the revised carrier.Source: §11 · p. 38, Figure 6 · p. 39

11.1 Persistent identity is memory, not only a mask

• Persistent identity is memory: merges affect future association, retrieval, and correction, so state should retain provenance, confidence, timestamps, supporting views, and alternative identities.DovSG demonstrates selective repair by relocalizing changed neighborhoods, removing obsolete voxels, fusing new observations, and regenerating affected graph edges, while residual features require explicit invalidation.Source: §11.1 · p. 39

11.2 Active observation closes the perception loop

• Active observation makes view selection part of segmentation, but systems must charge sensing and inference costs, test confirmation bias, and stop refinement using held-out evidence.OpenNeRF acquires novel views where projected language features disagree, CADRNet optimizes query-conditioned views, REALM degrades when refinement increases from 50 to 500 or 1000 steps, and Fun3DU separates language, localization, prompting, and agreement checkpoints.Source: §11.2 · p. 40

11.3 Relations expand the query interface

• Relation-capable carriers expand queries from unary labels to containment, support, reachability, affordance, and spatial structure, but unary masks, identities, relations, occupancy, and downstream success require separate evaluation.PairGS, ReLaGS, OGScene3D, Octree-Graph, Open-World 3DSG, and RelationField show that structured reasoning depends on grounded node and edge evidence rather than plausible answers alone.Source: §11.3 · p. 40

11.4 Query correction should be a first-class operation

• Correction should be a first-class operation that can revise names, split or merge entities with provenance, invalidate observations, and propagate only justified consequences across geometry, topology, semantics, and relations.EmbodiedSAM, ESAM++ and MoonSeg3R treat incoming masks as updates to persistent query state, but complete agentic memory still requires reversible identity decisions, confidence-aware forgetting, and actions that acquire disambiguating observations.Source: §11.4 · p. 41

12.1 Joint uncertainty over geometry, association, and semantics

• Future carriers should preserve factored uncertainty over geometry, projected support, teacher reliability, and competing identities, then propagate it to segmentation, view selection, and correction.Calibration should be tested under pose noise, missing views, domain shift, and teacher disagreement, not only in-distribution confidence.Source: §12.1 · p. 41

12.2 Dynamic and open-world identity maintenance

• Online systems should model identity as reversible state transitions that distinguish new objects, reappearance, motion, appearance change, and prior association errors.They also need identity confidence, bounded-memory retention of long-tail entities, and benchmarks with corrective queries and controlled changes.Source: §12.2 · p. 41

12.3 Generalizable carriers without semantic collapse

• Generalizable carriers should report scene, geometry, vocabulary, and task generalization separately while adapting lightly enough to retain rare structures without semantic collapse.Evaluations should separate visible from completed regions and quantify the accuracy-cost frontier between multimodal teachers and deployable students.Source: §12.3 · pp. 41–42

12.4 Carriers for overlap, hierarchy, and relations

• Carriers must represent overlapping affordances, materials, parts, object groups, hierarchies, and relations without making construction or querying quadratic.Evaluation should include overlapping ground truth and relation-sensitive queries because flat mIoU cannot measure these capabilities.Source: §12.4 · p. 42

12.5 Native-3D benchmarks and complete cost reporting

• Benchmarks should unify native surface semantics, multi-view annotations, identities, hierarchies, visibility, repeated prompts, latency, and complete reconstruction-to-query cost ledgers.Energy or accelerator-hours are essential for amortized models whose deployment cost otherwise hides omitted training expense.Source: §12.5 · p. 42

12.6 A unified correspondence audit

• A unified correspondence audit should hold geometry and 2D evidence fixed while comparing projection and association mechanisms through support, occlusion, boundary, and perturbation metrics.The audit would make apparent semantic gains interpretable, but implementing the shared benchmark remains future work.Source: §12.6 · p. 42

12.7 Hybrid automatic and interactive carriers

• Hybrid carriers should answer automatically from amortized memory, expose pseudo-label provenance, and request image-space correction only where uncertainty warrants interaction.The target combines revisable decomposition, open prompts, and direct deployment under identical shapes, prompts, and full data-engine accounting.Source: §12.7 · p. 42

Conclusion

• LAF represents 2D-to-3D transfer as decisions over generated evidence, geometric support, cross-view identity, conflicts, fusion, persistence, and querying, recorded through a carrier contract.The contract covers spatial support, semantics, identity, uncertainty, provenance, and supported operations.Source: §13 · p. 43
• Auditing 161 systems across six carrier families shows that association mechanisms can locate evidence without deciding identity, while carrier choices determine query and correction contracts.Point attributes, continuous fields, Gaussian state, object inventories, relation graphs, and temporal memories can all persist segmentation but expose different interfaces.Source: §13 · p. 43

Operational rules

• Correspondence and identity must be reported separately, because locating evidence on a surface does not establish which observations form an entity.This distinction applies across calibrated projection, renderer-mediated weights, learned correspondence, and temporary intermediaries.Source: §13 · p. 43
• Accuracy is meaningful only under a compatible output contract, and efficiency requires accounting for geometry, preprocessing, association, reconciliation, carrier construction, storage, and first and repeated queries.The cost ledger extends beyond the final inference step to include the full transfer pipeline.Source: §13 · p. 43
• Uncertainty and provenance must persist until correction is no longer needed, because irreversible proposals, support errors, or merges impose ceilings that downstream language reasoning cannot remove.Online, relational, dynamic, and feed-forward systems still require feedback and state versioning without eliminating the five decisions.Source: §13 · p. 43

Limitations and implications

• The validation is analytical rather than experimental: it establishes coverage and saturation on a frozen corpus but lacks prospective inter-annotator reliability and a controlled correspondence benchmark.These limitations define the next tests, while the broader design goal is coherent, accountable, and revisable persistent 3D perception.Source: §13 · p. 43