PaperlayerView exact revision on arXiv

Source-linked AI summary

Polarity--Time Interaction and Multi-Level Reliability for Event-Based Spacecraft Pose Estimation

Yonglong Zhang, Zongwu Xie, Yang Liu

arXiv:2610.36625v1cs.AIcs.RO

TL;DR

Sparse event responses create uneven keypoint errors whose effects on pose depend on correspondence geometry and the reliability of successive observations. The paper connects polarity–time localization with geometric recovery, event–CAD refinement, and quality-aware trajectory optimization. On the official SPADES challenge test set, its combined pose score is 0.0474, approximately 5.4% below PASAN+IRIS.

  • Problem

    Sparse event observations have uneven keypoint errors, and their pose effects depend on correspondence geometry and the reliability of successive observations.

    Source: §I · p. 1

  • Method

    The framework uses reliability-aware initialization, event–CAD refinement, and quality-aware kinematic smoothing to regulate observations across geometric recovery and trajectory estimation.

    Source: §I · p. 2

  • Results

    0.0474 combined pose score on the official SPADES challenge test set, approximately 5.4% below PASAN+IRIS.

    Source: B. Comparison with Existing Methods · p. 8, §V · p. 9

  • Takeaways & Limitations

    The framework links semantic correspondence initialization, event–CAD agreement, and time-aligned observation quality in a single pose-estimation pipeline.

    Source: §V · p. 9

    Continuous-sequence examples show persistent observation bias, and temporal weighting does not uniformly reduce every instantaneous error peak.

    Source: E. Qualitative Results · p. 9, §V · p. 9

Abstract

from arXiv · show

Sparse event responses produce uneven keypoint errors whose effects on spacecraft pose depend on correspondence geometry and the reliability of successive observations. We present a reliability-aware framework connecting polarity--time localization, geometric recovery, event--CAD refinement, and trajectory optimization. A three-branch representation combines positive events, negative events, and recent-event timing through cross-branch attention and gating, followed by spatiotemporal aggregation. An ordered semantic-point head estimates relative correspondence precision for geometrically constrained pose recovery. Event-edge and CAD-projection agreement then guide local pose refinement under keypoint-geometry safeguards. At the sequence level, a ten-dimensional descriptor combines heatmap shape, reprojection consistency, and geometric conditioning to set time-aligned observation scales. On the official SPADES challenge test set, the complete system achieves a combined pose score of 0.0474, approximately 5.4\% below PASAN+IRIS. Validation ablations reduce the score from 0.0772 for base localization with uniform PnP to 0.0514 for the complete cascade; on identical single-frame observations, quality weighting reduces the score from 0.0549 with constant-weight optimization to 0.0514.

I. INTRODUCTION

Event-based spacecraft pose estimation must handle sparse, spatially uneven observations whose effects depend on correspondence geometry and measurement reliability. The paper connects reliability-aware localization, geometric recovery, event–CAD refinement, and trajectory optimization.

I. INTRODUCTION

  • Relative spacecraft pose supplies geometric information for rendezvous, docking, on-orbit inspection, and space servicing.Event cameras report brightness-change locations, polarities, and times, providing fine-grained temporal measurements for dynamic targets.Source: §I · p. 1
  • The central challenge is that heatmap sharpness alone does not determine pose error, which also depends on camera rays, CAD-point configuration, refinement sensitivity, and final-observation quality.These dependencies link correspondence weighting, candidate acceptance, and trajectory weighting.Source: §I · pp. 1–2
  • The framework combines reliability-aware pose initialization, event–CAD consistency refinement, and multi-source quality-aware kinematic smoothing to regulate observations across the estimation stages.The proposed reliability mechanisms affect geometric recovery, refined-candidate acceptance, and trajectory optimization.Source: §I · p. 2

A. Visual Spacecraft Pose Estimation

  • Hybrid keypoint methods link learned observations to explicit target geometry, allowing localization and geometric recovery to be analyzed separately,.Related work also models correspondence weights probabilistically [11] and uses reliability to regulate measurements in navigation and visual odometry, [13].Source: A. Visual Spacecraft Pose Estimation · p. 2

B. Event-Based Spacecraft Pose Estimation

  • Prior event-based work covers learned semantic correspondences, model-projection matching, and temporal constraints, while this paper focuses on reliability at the interfaces between these operations,.The related work also includes the SPADES dataset and evaluation baselines [2] and compact event representations and neuromorphic deployment [5].Source: §I · p. 1, B. Event-Based Spacecraft Pose… · p. 3

III. METHOD

The method processes five time-adjacent event windows within smoothed target regions, then estimates semantic correspondences and pose through reliability-aware initialization and subsequent stages. Its polarity–time encoder combines positive-event, negative-event, and recent-event timing evidence.

A. Problem Definition and Overall Framework

  • Smoothed detection boxes define five adjacent event-window crops mapped to a common ROI grid, with retained inverse transforms mapping predicted keypoints back to sensor coordinates.Polarity and event-time cues establish local correspondences, while adjacent windows provide temporal context.Source: A. Problem Definition and Overall… · p. 3
  • RPI produces semantic points, point-level reliability, and an initial pose; ECR evaluates remaining event edges, and MQKS uses observation quality and heatmaps to recover the trajectory.The stages respectively establish initial geometry, refine the per-frame observation, and jointly estimate the trajectory.Source: A. Problem Definition and Overall… · p. 3

1) Polarity–Time Interaction Encoding:

  • Positive- and negative-polarity counts preserve opposite brightness-change support, while the most recent event-time channel indicates response recency at each location.Independent encoders process these three inputs before cross-branch interaction combines their structural and temporal evidence.Source: 1) Polarity–Time Interaction Encoding · p. 3
  • Six directed cross-attention edges connect the three branches, with target-derived queries and source-derived keys and values.All edges read pre-interaction features, so branch-update order does not change their information source.Source: 1) Polarity–Time Interaction Encoding · p. 3
  • Cross-branch attention exchanges information and channelwise gates regulate each branch’s contribution to the residual.The gates are predicted from the three global branch descriptors and normalized across branches.Source: 1) Polarity–Time Interaction Encoding · p. 3

F LCA

The description distinguishes within-window interaction from across-window aggregation and situates both in the full pose-estimation pipeline.

F LCA

  • The gated residual combines evidence within each window, while STA handles displacement across windows.The learned dual-polarity localization path is preserved, and the residual projection is zero-initialized.Source: F LCA · p. 3
  • The pipeline proceeds from detection and smoothed bounding-box ROIs through RPI and ECR to MQKS trajectory estimation.Solid arrows indicate estimation flow and geometric inputs; the dashed arrow carries auxiliary quality information.Source: Figure 1 · p. 4
  • RPI establishes semantic correspondences for an initial pose, ECR refines it using event–CAD agreement under geometric safeguards, and MQKS combines observations with time-aligned quality and motion constraints.Source: Figure 1 · p. 4

2) Spatiotemporal Aggregation and Keypoint Localization:

Spatiotemporal aggregation aligns neighboring-window features around a center window before keypoint localization. Heatmap responses and coordinates also provide descriptors for point reliability, while adaptation preserves inherited localization behavior and uses a single network at inference.

2) Spatiotemporal Aggregation and Keypoint Localization:

F STA

  • Swin-B and an FPN process aggregated features to predict 11 semantic-keypoint heatmaps, whose coordinates and local responses support reliability estimation.Together with camera rays and ordered CAD points, the descriptors determine relative precision for geometric refinement.Source: F STA · p. 4, Figure 2 · p. 5
  • Localization adaptation combines ground-truth Gaussian supervision with teacher-heatmap retention while protecting inherited peak locations and confidence-based point selection.The peak-safe set includes cells within the inherited decoded-error radius, the inherited argmax, and the cell nearest the ground-truth projection.Source: F STA · p. 4
  • During adaptation, the positive/negative encoders, dual-polarity fusion, STA, and localization tail are frozen; only temporal interaction components are updated.The teacher is used only during training, and inference requires a single localization network.Source: F STA · p. 5

3) Reliability-Guided Geometric Pose Recovery:

The reliability-guided recovery stage decodes continuous keypoints and uses heatmap, camera-ray, and CAD context to estimate correspondence precision. Those relative precisions weight robust pose recovery, while a pose-error propagation proxy complements localization-error supervision during reliability-head training.

3) Reliability-Guided Geometric Pose Recovery:

  • A bounded local Gaussian decoder fits a quadratic to log heatmap responses near the peak and falls back to the local centroid when curvature or offset checks fail.The decoded coordinate is mapped back to the sensor domain; the peak provides point-selection confidence.Source: 3) Reliability-Guided Geometric Pose… · p. 5
  • A 35-dimensional descriptor combines local evidence patches, ROI coordinates, camera rays, CAD coordinates, response maximum, and entropy before ordered cross-point context predicts a positive scale per point.Source: 3) Reliability-Guided Geometric Pose… · p. 5
  • Relative precision weights the robust pose objective initialized by EPnP-RANSAC, with bounded local updates and fallback rules for invalid initial estimates.A zero-initialized scale head reproduces the uniform-weight solver.Source: 3) Reliability-Guided Geometric Pose… · p. 5
  • The reliability head is trained in a separate stage with localization-error negative log-likelihood and a local pose-error propagation proxy.Source: 3) Reliability-Guided Geometric Pose… · pp. 5–6
  • The pose-propagation proxy uses projection Jacobians, prediction-derived support, and relative precision to penalize propagated dimensionless translation and rotation errors.Its target residual uses ground-truth coordinates, and its loss weight is frozen from gradient norms on 256 training examples.Source: 3) Reliability-Guided Geometric Pose… · p. 6
  • The initial semantic correspondence pose does not yet test consistency between remaining event edges and the CAD projection.The next stage uses those structures to supplement single-frame constraints.Source: 3) Reliability-Guided Geometric Pose… · p. 6

C. Geometrically Constrained Event–CAD Pose Refinement

Event–CAD consistency compares event-edge structure with rendered CAD projections using complementary orientation and spatial-distribution measures. Shared support, unsigned orientation, and unit-mass normalization make the comparison less sensitive to edge-sign reversal and magnitude differences.

C. Geometrically Constrained Event–CAD Pose Refinement

2) Candidate Selection under Geometric Constraints:

Candidate selection compares event-supported pose updates against a same-pool geometric reference and imposes geometric and integration-consistency checks. The subsequent quality descriptor combines heatmap, reprojection, and geometry information to set time-aligned observation scales for trajectory recovery.

2) Candidate Selection under Geometric Constraints:

  • Event-supported refinement uses a common local-perturbation pool and accepts candidates only when geometric bounds and event-agreement tests pass against the same-pool keypoint reference.Chronological and reverse-time surfaces guide proposals, while chronological orientation-moment scores determine final acceptance.Source: 2) Candidate Selection under Geometric… · p. 6
  • The selected candidate must be identical at two fixed area-integration resolutions, preventing gains attributable only to different integration precision.Source: 2) Candidate Selection under Geometric… · p. 6

1) Quality Description and Scale Calibration:

  • The pose-level quality descriptor combines heatmap response, reprojection consistency, and projected-geometry conditioning to distinguish observation contributions over time.Source: 1) Quality Description and Scale… · p. 7
  • The ten-feature descriptor includes geometric scales and conditioning, reprojection statistics, heatmap shape, and solver validity.The listed geometric entries are log translation and rotation scales plus normalized-information conditioning; reprojection entries encode median residual and the fraction of sufficiently confident, low-residual points.Source: 1) Quality Description and Scale… · p. 7
  • The geometric descriptor derives translation and rotation scales from a projection Jacobian and heatmap-moment pixel covariances, then inflates them according to reprojection residuals.This heatmap-moment proxy differs from the point-level task scale learned by RPI.Source: 1) Quality Description and Scale… · p. 7
  • After event refinement changes an observation, pose-dependent reprojection and geometry terms are recomputed while the four heatmap-shape terms are retained.The same ten-feature definition is used for calibration and inference.Source: 1) Quality Description and Scale… · p. 7
  • Translation and rotation error scales are calibrated separately using log-domain ridge regression on squared errors followed by radial scale correction.Runtime quality is computed from observations alone, with feature inputs and output scales clipped to fixed training-derived ranges.Source: 1) Quality Description and Scale… · p. 7
  • Calibrated scales enter observation residuals at matching physical timestamps, keeping each predicted scale aligned with its corresponding observation.Source: 1) Quality Description and Scale… · p. 7

2) Quality-Weighted Robust Motion Optimization:

Trajectory optimization retains individual frame observations and couples them through motion factors on consecutive translation and rotation increments. Quality scales control observation contributions, while invalid observations receive zero observation weight and are constrained through motion factors.

2) Quality-Weighted Robust Motion Optimization:

  • The trajectory states use additive translation and left-multiplicative rotation increments about final per-frame observations, initialized from smoothed translations and continuous relative rotation vectors.Source: 2) Quality-Weighted Robust Motion… · p. 7
  • Invalid observations receive zero observation weight, are interpolated for numerical initialization, and remain constrained by motion factors in sparse nonlinear least squares.A Huber penalty applies to each joint six-dimensional residual block.Source: 2) Quality-Weighted Robust Motion… · p. 7
  • Point-level reliability shapes each frame’s geometric solution, while pose-level quality controls the temporal contribution of each resulting observation.Source: 2) Quality-Weighted Robust Motion… · p. 7

IV. EXPERIMENTS

Experiments use SPADES spacecraft observations and separate validation ablations from the fixed final test evaluation. Comparisons use a fixed challenge metric and matched procedures for internal configurations.

A. Experimental Setup

  • SPADES provides Proba-2 event observations, camera calibration, a CAD model, and pose annotations for evaluation.The proposed method uses events as visual input and known camera–CAD geometry for pose recovery.Source: A. Experimental Setup · p. 7
  • Validation ablations use one predefined partition, with training and quality-calibration data separate from evaluation trajectories; test settings are fixed beforehand.Tables II–V cover cascade stages, localization and correspondence weighting, observation-quality weighting, and temporal context, respectively.Source: A. Experimental Setup · pp. 7–8
  • The combined challenge score is Epose = Et + ER, with translation and rotation errors below fixed thresholds set to zero and failures retained.Lower scores indicate better performance; the thresholds are 0.002173 for translation and 0.169° for rotation.Source: A. Experimental Setup · p. 8
  • The default front end uses five event windows, positive and negative count branches, recent-event timing, cross-branch attention and gating, and STA.It predicts 11 semantic heatmaps and relative correspondence precision for geometric recovery.Source: A. Experimental Setup · p. 8
  • Internal comparisons match data, model-selection rules, and training budgets; the default five-window system runs at 52.7 ms/frame.Quality mappings are calibrated for each observation stream and frozen during evaluation; external methods retain their published pipelines.Source: A. Experimental Setup · p. 8

B. Comparison with Existing Methods

The complete system achieves the lowest combined score among listed methods on the official SPADES test set. Validation studies report gains from the cascade, front-end changes, quality weighting, and five-window context.

1) Contributions of the Cascade Stages:

2) Three-Branch Interaction and Reliability-Guided Recovery:

  • 0.0772 falls to 0.0762, 0.0739, and 0.0729 as recent-event timing, cross-branch attention, and gating are added; reliability weighting then reaches 0.0697.With the interaction network fixed, NLL-based weighting gives 0.0716 before the pose-propagation proxy is added.Source: 2) Three-Branch Interaction and… · p. 8

3) Contributions of Multi-Source Quality:

  • Constant-weight optimization reduces the combined score from 0.0674 to 0.0549, while full MQKS reaches 0.0514—an approximately 6.4% reduction relative to constant-weight optimization.Heatmap-quality and geometry-quality optimization yield 0.0536 and 0.0522, respectively.Source: 3) Contributions of Multi-Source Quality · p. 8

B. Comparison with Existing Methods

D. Temporal Window Analysis

  • 0.0514 at five windows is the lowest tested combined score; one window gives 0.0677, while seven and nine windows give 0.0515 and 0.0523.Runtime rises from 36.8 ms/frame at one window to 52.7 at five, 61.3 at seven, and 70.2 at nine.Source: D. Temporal Window Analysis · p. 9

E. Qualitative Results

The full framework links event-based pose estimates to geometric refinement and time-aligned trajectory optimization. It achieves a combined pose score of 0.0474 on the official SPADES challenge test set, approximately 5.4% below PASAN+IRIS.

E. Qualitative Results

  • Figure 3 compares ground-truth and full-system estimated CAD bounding-box projections at four fixed timestamps across three preselected validation sequences.The projections use the same image coordinates; the display margin around the ROI does not change the network input or estimated pose.Source: E. Qualitative Results · p. 9, Figure 3 · p. 10
  • Figure 4 shows single-frame ECR pose observations before MQKS, projecting the object coordinate axes onto event ROIs across three sequences and four timestamps.The axes use a common physical length across panels.Source: E. Qualitative Results · p. 9, Figure 4 · p. 11
  • Across 599 timestamps in three examples, both temporal variants reduce rapid fluctuations, while MQKS modestly reduces mean rotation error relative to constant weighting but not every instantaneous peak.A sustained error around the middle of Example 2 remains visible, indicating residual observation bias.Source: E. Qualitative Results · p. 9, Figure 5 · p. 12

V. CONCLUSION

  • The framework connects polarity–time localization, point-level reliability, geometrically constrained event–CAD refinement, and observation-quality-aware trajectory optimization.Semantic correspondences initialize pose, refinement evaluates additional model-edge agreement, and the temporal stage regulates observations using time-aligned quality.Source: §V · p. 9
  • 0.0474 combined pose score on the official SPADES challenge test set, approximately 5.4% below PASAN+IRIS.The conclusion also reports a validation score reduction from 0.0772 to 0.0514 for the complete cascade, and a 6.4% reduction from quality weighting over constant-weight optimization.Source: §V · p. 9

DATA AVAILABILITY

Related papers

Loading 2610.36625v1…