PaperlayerView exact revision on arXiv

Source-linked AI summary

Human-in-the-Loop Geospatial Annotation for Rapid Dataset Construction in Field-Deployed UAV Systems

Morgan Masters, Adam Korycki, Nikolaas Bender, T. Luca Altaffer, Nick Kuipers, Colleen Josephson, Steve McGuire

arXiv:2609.28767v4cs.RO

TL;DR

Manual image annotation is costly, while available aerial datasets and georeferencing results do not meet the needs of some fine-scale field targets. BirdsEye records targets in the field and projects them into UAV images, with an uncertainty model for annotation and sensor design. At a geographically distinct test site, trained configurations recovered 0.563–0.887 of in-view geotags.

  • Problem

    Manual image annotation is time-consuming, and existing aerial data and georeferencing accuracy can be insufficient for fine-scale field targets.

    Source: §1 · p. 2

  • Method

    BirdsEye records target locations in the field using RTK positioning, projects them into UAV images, and models pose-to-pixel uncertainty for sensor-noise design.

    Source: §1 · p. 2, §5 · p. 20

  • Results

    At the held-out site, configurations recovered 0.563–0.887 of geotags in view, and recall rose by an average of 0.231 across configurations.

    Source: §5 · p. 22

  • Takeaways & Limitations

    In the reported three-site case study, two workers annotated 12,524 frames and 55,600 labels in roughly 12 hours, a 25.5× per-worker increase over manual image annotation.

    Source: §5 · p. 22

    The planar scene approximation’s operating envelope is limited: terrain-induced reprojection error exceeds motion-induced error at about 10° slope and increases rapidly beyond it.

    Source: §5 · p. 21

Abstract

from arXiv · show

Real-world perception systems must adapt to changing environments, but manual image annotation cannot scale to field data volumes. We present BirdsEye, which shifts expert annotation from images to the field: an operator records target locations in world coordinates using RTK positioning and calibrated projective geometry propagates each observation to all frames where the target is visible. To quantify how well physical annotations align with image observations, we derive a first-order mapping from camera-pose uncertainty to pixel uncertainty and validate it against Monte Carlo simulation. This mapping is linear in the six per-axis pose variances, so it inverts into a sensor design tool: we give a sufficient condition converting an annotation tolerance into a convex set of admissible pose-noise budgets, a closed-form largest admissible scaling of a deployed sensor suite, and a unique per-axis pose specification under an equal-budget-share allocation. We also analyze the planar-surface approximation underlying the projection, which holds up to 10 degrees of terrain slope. By direct measurement, we show that system projection accuracy is sub-decimeter (sub-30 pixel) at AGL altitudes of 10-20m under conditions excluding sustained yawing. During an in-field case study across three agricultural sites, two field workers produced 12,524 annotated frames carrying 55,600 labels in roughly 12 hours (25.5x per-worker rate increase over manual labeling). Detectors trained on imagery collected by this workflow recovered 56-89% of in-view surveyed targets at a geographically distinct farm, at pre-registered operating points; human review of the leading configuration estimates detection precision at 83-87%, spanning three tie-break conventions for clusters carrying contradictory human verdicts.

1. Introduction

BirdsEye addresses the mismatch between image-centered annotation and field experts’ ability to identify small targets in their physical environment. It records targets in world coordinates, projects them into UAV imagery, and models how pose uncertainty affects image-space labels.

1. Introduction

  • Existing aerial datasets and georeferencing results often lack the spatial fidelity needed for millimeter-to-centimeter targets viewed from low-altitude UAVs.The cited direct-georeferencing accuracies of 8–11 cm are inadequate for targets a few millimeters across.Source: §1 · p. 2
  • BirdsEye records target locations in the field using RTK positioning, then projects them into UAV images through calibrated camera geometry.The same geometric representation can map image-space detections back into world coordinates.Source: §1 · p. 2
  • The workflow separates identifying a target from associating it with individual images, automating the latter through calibrated projective geometry.This shifts annotation from repeated image-by-image labeling to field-centric observation.Source: §1 · p. 2
  • A first-order model maps camera-pose uncertainty to pixel uncertainty and inverts that relationship into admissible sensor-noise budgets.The sufficient design condition defines a convex set of budgets, a closed-form scaling for a deployed suite, and a unique equal-contribution per-axis specification; it is conservative by at most a factor of two.Source: §1 · pp. 2–3
  • Across field deployments, the evaluation uses a geographically distinct held-out site with operating points fixed before scoring and detections reviewed by a human.The campaign comprised 46,205 aerial frames and 122,467 field annotations across two deployment sets.Source: §1 · p. 3

2. Background and Related Work

Prior annotation approaches generally keep images as the fundamental unit, even when they use geographic context or reduce manual labeling. BirdsEye instead starts from physical-world observations and uses geometry to associate them with images.

2. Background and Related Work

  • Image annotation platforms and weakly supervised or learned methods reduce labeling effort, but their workflows remain centered on image-space observations.Image-centered systems are effective when the image is the natural domain for identifying a target, whereas field experts may have better access to targets in the physical environment.Source: §2 · p. 4

2.1. Geospatial and Image-Based Annotation

  • BirdsEye differs from georeferenced-image approaches by making a physical-world observation, rather than an image, the fundamental annotation event.It propagates that observation to every image in which the target is geometrically visible.Source: §2.1 · pp. 4–5

2.2. Synthetic Datasets

  • Synthetic datasets can provide abundant labeled observations, but transfer from simulated to observed imagery is a principal limitation for field-deployed remote sensing.BirdsEye uses real-world imagery and avoids requiring a separate synthetic representation of the target domain, but does not eliminate data-collection costs.Source: §2.2 · p. 5

2.3. Physical and Robotic Label Generation

  • Prior physical-labeling systems use markers or indoor robotic workspaces, while BirdsEye uses RTK-referenced field observations for outdoor UAV surveys.Its calibrated camera model associates the same physical annotation with visible observations across a moving aerial platform.Source: §2.3 · p. 5

2.4. Pixel Georeferencing

  • Georeferencing assigns geographic coordinates to image pixels, with ground-control-point methods and direct georeferencing as two dominant paradigms.Ground-control-point methods estimate a transformation from correspondences between image pixels and surveyed landmarks.Source: §2.4 · p. 6, §2.4.1 · p. 6

2.4.1. GCP-Based Georeferencing

  • A reported ground-control-point accuracy of 3.5 cm used 10–12 points per km2, but the study’s 1.7 cm GSD was inadequate for ultra-fine-grain tracking.The passage describes GCPs as visually identifiable landmarks with high-accuracy ground-truth coordinates.Source: §2.4.1 · p. 6

2.4.2. Direct Georeferencing

  • Direct georeferencing uses onboard localization sensors to estimate camera pose at capture, with reported accuracies of 8.7 cm and 11 cm ± 2.14 cm.Those studies used imagery with several-centimeter GSD, while sub-3 mm GSD requires higher precision to avoid pixel-level misalignment.Source: §2.4.2 · p. 6

3. Materials and Methods

The methods combine RTK field annotation, UAV sensing, calibrated projective geometry, and a structured survey protocol. They also specify inclusion and flight-coverage practices to limit label ambiguity and false negatives.

3.1. The BirdsEye Geoannotation Hardware

  • The field annotation hardware uses RTK basestation and handheld rover units, with commercially available components and an open-source NTRIP correction service.The basestation broadcasts RTCM3 corrections by radio and provides a local Wi-Fi NTRIP server.Source: §3.1 · p. 6

3.2. Airframe and Payload Sensors

  • The DJI M300 RTK carries a Raspberry Pi 5, 2MP GigE camera, radar altimeter, and RTK GPS-provisioned 9-DoF INS.These components provide visual sensing, altitude, and inertial navigation data.Source: §3.2 · p. 6

3.3. Payload Calibration

  • Camera intrinsics and distortion are estimated from calibration imagery, then combined with inertial data to estimate the rigid transform between INS and cameras.The resulting calibration parameters are fixed and assumed constant thereafter.Source: §3.3 · p. 7

3.4. Payload Synchronization

  • Exposure-start strobe pulses trigger INS pose captures, and image-pose pairs are matched using a framerate-based timing heuristic.Pairs exceeding the reciprocal-framerate age difference are discarded, and matched images are removed from the candidate set to avoid double-matches.Source: §3.4 · p. 7

3.5. Field Operations Protocol

  • Annotators survey partitioned field areas by placing a handheld GPS antenna above each target and recording coordinates and metadata before autonomous UAV flights.A shortest-path flight planner uses the survey records, and the drone captures imagery at 5 Hz.Source: §3.5 · p. 7

3.5.1. Defining Field Survey Inclusion Criteria

  • Agreed inclusion criteria and annotator training are used to keep target labels consistent when burrow appearances are ambiguous.The paper identifies mound visibility and target degradation as sources of ambiguity.Source: §3.5.1 · p. 8, Figure 3 · p. 8

3.5.2. False Annotation Mitigation

  • Flying outside surveyed regions or missing targets during annotation creates false negatives in the resulting dataset.The protocol controls flight coverage and notes that missed targets complicate simple definitions of precision.Source: §3.5.2 · p. 8

3.6. The BirdsEye Software Backend

The backend projects surveyed world-coordinate targets into UAV imagery under a planar scene approximation, then handles visibility, uncertainty-aware labels, and detection georeferencing.

3.6.1. Automatic Annotation Process

  • Projection assumes a planar surface and maintained calibration and RTK, while target height and local relief can shift projected points, motivating true-geometry ray casting for future subcanopy or infrastructure use.At d = 10 m and r ≤ 2.5 m, 52 cm of relief displaces a projected point by roughly 13 cm; the campaigns maintained RTK under open sky, while mobile targets and degraded fixes fall outside the assumptions.Source: §3.6.1 · pp. 8–9

3.6.2. Geotag Detection

  • A geotag is in-frame when its projected location passes a cross-product sign test against the frame-boundary segments.Figure 4 illustrates the world-to-image geometry and the edge and tag vectors used in the test.Source: §3.6.2 · p. 9, Figure 4 · p. 10

3.6.3. Noise-Aware Annotations

  • Gaussian soft labels place 3-σ patches around in-frame geotags, using empirical or propagated uncertainty to set σ; wider patches improve signal coverage but dilute reward and increase nonzero pixels as radius².The case study used σ = 10 px, slightly below the measured mean empirical standard deviation of 13.55 px.Source: §3.6.3 · p. 9

3.6.4. Detection Georeferencing

  • Detections are back-projected into world-frame rays and intersected with the local planar surface to estimate positions; recall and geotag-matched cluster fraction (GMCF) quantify detection accuracy before and after human review.Source: §3.6.4 · p. 10

3.7. Bounding Admissible Sensor Noise

The section turns first-order pose-to-pixel uncertainty into sensor-noise design constraints and deployable pose budgets. It derives a sufficient tolerance bound, a closed-form suite scaling, and an equal-share per-axis specification.

3.7. Bounding Admissible Sensor Noise

  • A first-order se(3) pose-perturbation model propagates pose covariance into pixel covariance, and Monte Carlo checks the linearization’s validity regime.Source: §3.7 · p. 10

3.7.1. A Design Bound Linear in the Sensor Noise Budget

  • The projected covariance is rank 2 and positive semidefinite, and its trace is linear in pose-variance entries; the estimate has a factor-of-two error bound.Source: §3.7.1 · pp. 10, 12
  • The sufficient annotation-tolerance bound is linear in six per-axis pose variances, forming a convex polyhedron whose worst-imaged point reduces the frame-wide constraint to one linear row.Source: §3.7.1 · p. 12
  • Positive homogeneity gives a closed-form largest uniform scaling of a deployed sensor suite that remains admissible.Source: §3.7.1 · p. 12

3.7.2. Inversion to a Per-Axis Pose Budget

  • The feasible six-dimensional budget is non-unique; maximizing geometric mean selects a unique interior allocation, while equal budget share assigns each degree of freedom the same worst-case pixel variance.Maximizing total variance instead concentrates the entire budget on one axis, which the text identifies as an unusable specification.Source: §3.7.2 · p. 12

4. Results

BirdsEye combines field geotagging with camera projection and uncertainty analysis, then evaluates projection accuracy, dataset construction, and detection on a geographically distinct site. Results include a 25.5× annotation-rate increase and terrain- and pose-related limits on projection.

4.1. Empirical Characterization of Projection Error

  • Single-digit-centimeter mean and standard-deviation geospatial projection errors were measured at 10 m and 20 m, excluding sustained yawing.The validation used AprilTag pixel-space and RTK world-frame ground truth, with static hovers, yawing, and overflights among the maneuvers.Source: §4.1 · pp. 12–13, §4.1.2 · p. 13, §4.1.1 · p. 13

4.2. Annotation Sensitivity to Pose Uncertainty

  • At unit pose-noise scale, the analytical uncertainty model predicts a major axis of roughly 15 px or less across simulated altitudes, with larger values on lower flights.Analytical and Monte Carlo results agree strongly; divergence occurs above scale factors of 10 at 5 m AGL, while altitude uncertainty dominates at low altitudes.Source: §4.2 · pp. 13–14, Figure 6 · p. 14

4.3. Applying the Sensor Noise Analysis

  • At 10 m, the trace bound rejects the deployed budget at κ⋆ = 0.9207, while the exact eigenvalue constraint accepts it at κ⋆ = 1.335.At 20 m, the trace-bound κ⋆ is 2.154; the authors recommend trace-based allocation and screening, with exact confirmation near the tolerance boundary.Source: Table 2 · p. 14, §4.3 · p. 14

4.4. Annotation Sensitivity to Terrain Slope

  • Around 10° of terrain slope, terrain-induced error exceeds pose-induced error; the flat-world approximation is treated as nominally valid up to roughly that slope.Beyond 10°, error increases rapidly; measured georeferencing errors were approximately 8 cm at 10 m AGL and 18 cm at 20 m AGL.Source: §4.4 · pp. 15–16, Figure 8 · p. 16

4.5. Deployment Case Study: Burrowing Mammals

  • 25.5× was the average per-worker annotation-rate increase over GUI-based labeling.Two field workers conducted the campaign across three sites; the productivity estimate depends on camera framerate, which was 5 Hz.Source: §4.5 · pp. 16–17, Table 4 · p. 17

4.6.1. The Detector and Training Protocols

  • The held-out detector used 224×224 subtiles, Gaussian soft labels, and operating points selected on validation flights before the unseen-site test.The detector was an evaluation instrument for dataset transfer, not an architecture comparison or recommended checkpoint.Source: §4.6 · p. 17, §4.6.1 · pp. 17–18

4.6.2. Held-out Results

  • On the unseen site, recall rose by 0.231 and GMCF fell by 0.176 on average across 12 configurations; validation-to-test F1 rank correlation was +0.007.Recall is transferable across survey densities, but GMCF and F1 are not comparable across sites with different survey density.Source: §4.6.2 · p. 18, Table 6 · p. 19

4.6.3. Adjudication

  • 0.522 PRE GMCF on the max-F1 arm becomes 0.832–0.867 POST depending on the tie-break convention.POST values are reported under three conventions because contradictory verdicts are not resolved by data.Source: Table 7 · p. 20, §4.6.3 · p. 20

5. Discussion and Conclusion

BirdsEye combines field-based geospatial annotation with uncertainty analysis and sensor design bounds. Field deployments show rapid annotation and transferred detection recall, while the discussion identifies geometric and evaluation boundaries.

5. Discussion and Conclusion

  • 12,524-frame deployment produced sub-decimeter projection errors at 10–20 m AGL, with the largest errors during sustained yawing maneuvers.These measurements support sufficiently precise image correspondence for the demonstrated agricultural task.Source: §5 · pp. 20, 22
  • The first-order uncertainty model closely matched Monte Carlo over operational pose perturbations, with altitude uncertainty especially influential at lower operating altitudes.At sufficiently large perturbations, higher-order effects cause the analytical approximation to diverge from Monte Carlo.Source: §5 · p. 20
  • The linear pose-variance model turns annotation tolerance into a sensor-design bound, enabling single-inner-product suite screening and a closed-form uniform scaling.The equal-share per-axis allocation is scale invariant; its tolerances depend on imaging geometry and the annotation tolerance, not the deployed platform.Source: §5 · pp. 20–21
  • The trace-based design condition is sufficient but not tight: at 10 m it rejects the deployed budget by 8%, although the exact eigenvalue condition accepts it with 1.34× headroom.At 20 m, the two conditions agree; platform conformance is decided by Equation 4, not by treating each Table 3 allocation as a pass/fail threshold.Source: §5 · p. 21
  • Terrain-induced reprojection error exceeds motion-induced error at about 10° slope and rises rapidly beyond, bounding the locally planar approximation.The authors identify explicit 3D geometry and ray-surface intersection as a direct extension that removes this principal geometric approximation.Source: §5 · pp. 21–22
  • 12,524 frames and 55,600 annotations were produced across three sites in roughly 12 hours, a 25.5× per-worker throughput increase over the study’s manual image-annotation baseline.The comparison concerns the demonstrated end-to-end workflows, not human annotation speed in general.Source: §5 · p. 22
  • 0.563–0.887 of in-view geotags were recovered on the geographically distinct held-out flight, supporting dataset sufficiency for modest models rather than claims about detector architecture.GMCF is not comparable across sites because the validation flights were surveyed 3.253× more densely than the test flight.Source: §5 · p. 22
  • The field study primarily evaluates a manual-annotation baseline and a single agricultural detection task, so wider application requires comparisons across strategies, sensors, terrain, and target classes.The geoannotation hardware uses commercially available components and free correction software rather than bespoke instrumentation.Source: §5 · p. 22

Appendix A. Propagation of Pose Uncertainty into Pixel Space

Appendix A formulates projective error under pose perturbations and uses first-order propagation to estimate pixel-space covariance. It derives the projection Jacobian used later for design bounds and Monte Carlo validation.

Appendix A. Propagation of Pose Uncertainty into Pixel Space

  • The projected point is formed by perspective division of camera-frame coordinates using the camera intrinsics and the world-to-camera transform.The appendix identifies the world point, camera-frame point, unnormalized projection, and projection operator used in this mapping.Source: Appendix A · p. 23
  • Pose noise is injected as a left perturbation of the nominal pose, with a six-dimensional Gaussian vector whose diagonal covariance entries come from sensor datasheets.The six-vector is partitioned into translation and rotation components.Source: Appendix A · p. 23
  • The analysis defines projective error as predicted minus true image coordinates and approximates it by discarding second- and higher-order Taylor terms.Under the stated assumption that the nominal projection equals ground truth, the remaining first-order term drives the uncertainty estimate.Source: Appendix A · p. 24
  • Projected-coordinate covariance is approximated as JξΣ0Jξᵀ, propagating pose covariance through the projection Jacobian.The Jacobian is derived by the chain rule for normalized projected coordinates.Source: Appendix A · p. 24
  • The resulting Jacobian uses the first-order exponential-map approximation, then supports design-bound derivation and Monte Carlo validation.The appendix identifies rotation and translation in the pose transform and points to Sections 3.7 and 4.2 for those applications.Source: Appendix A · p. 24

Appendix B. Detection Metrics and Their Estimands

The appendix defines geospatial detection metrics around surveyed targets and cross-frame detection clusters, distinguishing geometric matching from human adjudication. It also explains why the resulting quantities have different estimands and why conflicting cluster judgments require explicit tie-break conventions.

Appendix B. Detection Metrics and Their Estimands

  • Precision is measured over cross-frame detection clusters formed after back-projection, 3 cm-grid deduplication, and DBSCAN clustering.Cluster identity depends on ε and m, so measurements at different clustering parameters are not comparable.Source: Appendix B · p. 25
  • Recall counts in-view RTK geotags recovered within εpred of a cluster core point; 151 of 161 held-out-flight geotags were in view.GMCF is the fraction of clusters containing at least one in-view geotag, expanded after adjudication to clusters with a confirmed-positive verdict.Source: Appendix B · p. 25
  • GMCF is a lower bound on precision, not precision itself, and is valid for relative comparison within one site but not across sites.The survey may omit real burrows, and the bias scales with survey density.Source: Appendix B · p. 25
  • PRE recall assumes survey omission is independent of detectability, while POST recall enlarges the positive class through human adjudication.The two arms therefore cannot share one recall denominator.Source: Appendix B · p. 25
  • POST recall is reported as a strict upper bound when no added markers are placed, because adjudication can discover positives only where the detector fired.The reported values are marked with ≥ because the adjudicated positives are censored.Source: Appendix B · p. 25
  • The paper does not report geospatial F1 because cluster-based precision and geotag-based recall have different denominator units and meanings.A cluster may absorb several geotags, making their harmonic mean a category error.Source: Appendix B · p. 25
  • A cluster can contain contradictory human judgments, and positive-wins, majority-wins, and hard-negative-wins conventions produce the reported spread.A cluster may merge distinct spots a few decimeters apart under reviewer-label propagation.Source: Appendix B · pp. 25–26

Appendix C.1. Checkpoint Screening

Checkpoint screening applies two independent health guards before ranking the validation results. The appendix reports which checkpoints passed, how the cuts were chosen, and the operational spread among healthy checkpoints.

Appendix C.1. Checkpoint Screening

  • 11 of 30 checkpoints passed both health guards; 11 were classified as collapsed and 8 as too sparse to score.The screening used 3 validation flights, a fixed confidence threshold of 0.03, and 1,500 frames in total.Source: Appendix C.1 · p. 26, Figure A1 · p. 26
  • Among the 11 healthy checkpoints, the field spans 67× in review load for 3.0× in recall, an operational trade rather than a ranking.The table explicitly does not recommend a row.Source: Table A1 · p. 27
  • The distinct-position guard rejects checkpoints below 0.05, while the detection-rate guard rejects those below 1.7 detections per frame on their worst validation flight.The guards test independent failure modes; neither subsumes the other.Source: Appendix C.1 · p. 26, Figure A2 · p. 27
  • The two cut values are sample-based decisions about neighboring checkpoints, with Figure A2 showing the margins to the nearest checkpoint on each side.Only checkpoints bounding each cut are labeled in the figure because their values determine where the cut may sit.Source: Figure A2 · p. 27

Appendix C.2. Operating-Point Sweep

The operating-point sweep filters configurations for credible cluster geometry before comparing their scores.

Appendix C.2. Operating-Point Sweep

  • 3,997 of 14,400 swept configurations passed the geometric credibility filter, while 1,484 produced no hit cluster.Without the filter, chained clusters can inflate GMCF and F1 while merging many geotags across tens of meters, making them unsuitable for locating a burrow.Source: Appendix C.2 · p. 27

Appendix C.3. Pre-registration

The held-out evaluation uses operating points selected and sealed before detections existed for that flight.

Appendix C.3. Pre-registration

  • The pre-registered selection rule fixed 3 confirmatory arms, each with a checkpoint and complete operating point, and covered 12 configurations across 11 checkpoints.Other configurations were pre-registered as descriptive.Source: Appendix C.3 · p. 28

Related papers

Loading 2609.28767v4…