CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

• Camera appearance features become unreliable under occlusion, while LiDAR tracking can mismatch objects during irregular motion or large inter-frame displacements; fusion methods also face false detections and cross-category identity switches.Source: §I · pp. 1–2
• CAMO-MOT combines image and point-cloud information through O2S-OAM, CS-MM, and M2-FAM to address occlusion, false detection, and intra-class association problems.Source: §I · p. 2
• O2S-OAM selects appearance features according to estimated occlusion state, while CS-MM raises motion costs for low-confidence detections and M2-FAM restricts association to the same category.CS-MM uses a 3D generalized intersection-over-union distance criterion; M2-FAM constructs a multi-category loss.Source: §I · p. 2
• CAMO-MOT ranks first on KITTI multi-modal MOT (+5.60% HOTA, +2.56% MOTA vs. EagerMOT; lowest IDS: 23 Car, 137 Pedestrian) and nuScenes (+1.20% AMOTA vs. BEVFusion).Source: §I · p. 2

II. RELATED WORK

• Prior 2D and 3D tracking approaches face complex-scene failures, occlusion sensitivity, modality-alignment challenges, and limited handling of multiple categories.The cited discussion notes that point-cloud sparsity can impede effective image–point-cloud alignment and that many methods lack occlusion branches.Source: §II · pp. 2–3
• CAMO-MOT introduces an occlusion head and multi-category loss, processing the two modalities independently through two associations.Source: §II · p. 4

III. CAMO-MOT

• CAMO-MOT follows tracking-by-detection with O2S-OAM, CS-MM, and M2-FAM; motion association is applied first, followed by appearance association for remaining detections and trajectories.Source: Figure 2 · p. 3, §III · p. 4

A. Optimal Occlusion State-based Object Appearance Module (O2S-OAM)

• A modified DLA-34 extracts image feature maps from the image at moment t.The maps are denoted F_t, with M representing the number of feature maps.Source: A. Optimal Occlusion State-based… · p. 4
• The matching head uses feature differences between two frames to produce an association matrix for object matching.The matrix is normalized separately by row and column, with added entries for objects entering or leaving the scene.Source: A. Optimal Occlusion State-based… · p. 4
• The occlusion head identifies each object's occlusion state so the tracker can select its best appearance feature across multiple moments.Object image areas are bilinearly resized to 224×224; the resulting Occs term is used for online appearance-feature selection.Source: A. Optimal Occlusion State-based… · p. 4

B. Confidence Score-based Motion Module (CS-MM)

• Low-confidence detections receive higher costs, and trajectories with lower state prediction scores incur higher motion-association losses.The motion cost matrix is divided by the state score, which represents trajectory prediction reliability.Source: B. Confidence Score-based Motion… · pp. 4–5
• The Kalman-filter state model uses the previous tracked state and a state transformation to estimate the current state and its error covariance.The state includes 3D position, velocity, acceleration, object dimensions, and rotation; velocity information depends on the dataset.Source: B. Confidence Score-based Motion… · p. 5
• The motion cost matrix uses gIoU3D and current-frame 3D detections containing position, size, rotation, and, when available, ground-plane velocity.The passage identifies the matrix as the motion cost calculated with gIoU3D.Source: B. Confidence Score-based Motion… · p. 5

C. Multi-category Multi-modal Fusion Association Module (M2-FAM)

• A category loss is added to motion and appearance association costs so objects are associated only within their categories.Category indicators are separated by a large gap to avoid inter-class interference in subsequent associations.Source: C. Multi-category Multi-modal Fusion… · p. 5
• The module associates trajectories by motion first, then applies appearance association to the remaining detections and trajectories.Motion and appearance associations use their respective maximum-cost thresholds.Source: C. Multi-category Multi-modal Fusion… · p. 5
• Appearance features and occlusion states are retained over time, and lower-occlusion-state features are selected when updating trajectories.The paper states that features containing information beyond the object itself are unreliable under occlusion.Source: C. Multi-category Multi-modal Fusion… · p. 5

D. Training

• Training image pairs use a random interval of 1 to nmax frames to promote robustness to temporary occlusions or missed detections.ResNet18 is the occlusion-head backbone, trained with cross-entropy loss.Source: D. Training · pp. 5–6
• The ground-truth association matrix has size (Nmax + 1) × (Nmax + 1), with its extra row and column marking lost or newly emerged objects that should not be associated.Entries indicate whether objects are associated.Source: D. Training · p. 6
• The final joint loss combines the CenterNet detection-head loss with learnable weights for detection and other branches, including the occlusion and matching heads.The matching loss averages the association losses for the two input frames.Source: D. Training · p. 6

IV. EXPERIMENTS

• Experiments cover KITTI 2D MOT, KITTI 3D MOT, and nuScenes 3D MOT, with qualitative visualizations also included.The section presents benchmark comparisons and ablation studies in addition to the visualizations.Source: §IV · p. 6
• The KITTI training set has 8,008 frames, while its test set has 11,095 frames; the stated validation split uses sequences 1, 6, 8, 10, 12–16, 18, and 19.The KITTI tracking benchmark provides 21 training sequences and 29 test sequences.Source: §IV · p. 6
• KITTI evaluation uses HOTA as the primary 2D MOT metric and AMOTA under the cited protocol for 3D MOT.KITTI 2D tracking follows CLEAR MOT and HOTA; the 3D protocol follows.Source: §IV · p. 6
• nuScenes provides keyframes at 2 Hz, and its official 3D tracking evaluator uses AMOTA as the primary metric.The dataset has 850 training sequences and 150 test sequences; its low keyframe frequency entails substantial inter-frame displacement.Source: §IV · p. 6
• HOTA accounts for tracking performance across detection thresholds, while MOTA and MOTP are CLEAR-MOT metrics.The paper describes MOTP as measuring the overlap between estimated objects and ground truth and also reports MT, ML, FP, FN, and IDS.Source: §IV · pp. 6–7
• For KITTI 3D MOT and nuScenes 3D MOT, evaluation follows averaged MOTA and MOTP across recall thresholds, with scaled variants used to range from 0% to 100%.The paper notes that the official nuScenes AMOTA is sAMOTA in.Source: §IV · p. 7

A. Implementation Details

• The implementation extracts 13 feature maps from a modified DLA-34 backbone and sends the final map to the occlusion head.Training data are augmented with random cropping, flipping, scaling, and photometric distortions.Source: A. Implementation Details · p. 7
• On KITTI, CAMO-MOT trains for 100 epochs with Adam, batch size 8, and initial learning rate 1.25e−4.The learning rate is decreased at 60 epochs; inference runs at 25 FPS per image on an RTX 3090 GPU.Source: A. Implementation Details · p. 7
• KITTI inference uses Hungarian matching thresholds of θmo = 0.01 and θApp = 1.4, and discards trajectories not updated for 15 frames.Hyperparameters are selected using validation HOTA; unmatched detections initialize new trajectories.Source: A. Implementation Details · p. 7
• For nuScenes, training uses Adam for 60 epochs at learning rate 1.25e−4 with batch size 8.Inference uses detector-specific confidence filters, projects each 3D box onto six cameras, and uses the largest projected 2D box for image features.Source: A. Implementation Details · p. 7
• nuScenes inference allows up to two predicted frames during each mismatch period and deletes trajectories after 15 unupdated frames or when their score falls below the deletion threshold.The predicted state score is 0.05 times the score of the most recently updated state.Source: A. Implementation Details · p. 7

B. Comparative Evaluation

• Car and Pedestrian IDS are 23 and 137, respectively, the lowest among methods; versus EagerMOT, HOTA rises by 5.56% and 5.52% and MOTA by 2.56% and 2.37%.The comparison uses the identical detector for CAMO-MOT and EagerMOT.Source: B. Comparative Evaluation · pp. 7–8, Table I · p. 8, Table II · p. 8
• On KITTI 3D MOT, sAMOTA increases by 2.01% for Car and 13.98% for Pedestrian versus AB3DMOT.The evaluation is on the KITTI validation set.Source: B. Comparative Evaluation · p. 8, Table III · p. 8
• 75.3% AMOTA places CAMO-MOT first among algorithms on the nuScenes test set, 1.20% above second-ranked BEVFusion [20].The comparison is reported despite CAMO-MOT using a lower detector than BEVFusion.Source: B. Comparative Evaluation · p. 8, Table V · p. 9
• Across multiple detectors, CAMO-MOT reports 80.74% Car HOTA and 50.24% Pedestrian HOTA with PointGNN, and 76.3% AMOTA with fused BEVFusion [20] and FocalsConv.The authors report that the method adapts to different detectors; on nuScenes it also obtains minimal IDS without losing AMOTA precision.Source: B. Comparative Evaluation · pp. 8–9, Table IV · p. 8, Table VI · p. 9

C. Ablation Studies

• For Car, fusion with the occlusion head improves HOTA by 1.46%, MOTA by 0.71%, and IDS by −16 versus fusion alone; for Pedestrian, the changes are +0.48%, +0.32%, and −8.Fusion alone also improves all three reported measures over point-cloud-only and image-only methods in both classes.Source: C. Ablation Studies · p. 9, Table VII · p. 9
• Optimal-occlusion features (OCC) track better than last-three-frame features (LTF) or their occlusion-weighted combination (LTF+OCC).The reported occlusion weights are 1, 0.7, 0.3, and 0 for fully visible, partly occluded, largely occluded, and fully occluded states.Source: C. Ablation Studies · pp. 9–10, Table VIII · p. 10
• Association order changes KITTI results: MO→AP and AP→MO yield HOTA gains of 0.35% and 0.55%, MOTA gains of 0.18% and 0.38%, and IDS reductions of 15 and 27 for Car and Pedestrian, respectively.The paper suggests similar appearances may contribute to identity switches.Source: C. Ablation Studies · p. 10, Table IX · p. 10
• Adding category loss raises HOTA by 0.93% for Car and 0.25% for Pedestrian compared with no category loss.The authors say category loss eliminates the possibility of associations between categories.Source: C. Ablation Studies · p. 10

D. Visualization

• The section reports qualitative visualization experiments comparing single and fused modalities on KITTI.Source: D. Visualization · p. 10
• Across scenes 1–5, CAMO-MOT avoids new identities during occlusion, sudden acceleration, and turns, but only partly resolves severe occlusion with irregular motion.The method improves on single modalities in scene 5; the passage also characterizes tracking as stable under severe occlusions.Source: D. Visualization · p. 10
• Table X is labeled as a category-loss ablation contrasting no category loss (NCL) with category loss (CL).Source: Table X · p. 10
• CAMO-MOT maintains stable tracks through occlusion where DEFT and EagerMOT switch identities, acceleration where PC3T switches, and illumination changes where JMODT starts a new track.These comparisons are shown against DEFT, EagerMOT, JMODT, and PC3T on KITTI.Source: D. Visualization · p. 10

V. CONCLUSION

• CAMO-MOT achieves state-of-the-art performance among multi-modal methods on KITTI and all methods on nuScenes, with the lowest IDS among KITTI algorithms and a top-ten IDS result on nuScenes.Source: §V · p. 10