PaperlayerView exact revision on arXiv

Source-linked AI summary

ToxTrac: a fast and robust software for tracking organisms

Alvaro Rodriquez, Hanqing Zhang, Jonatan Klaminder, Tomas Brodin, Patrik L. Andersson, Magnus Andersson

arXiv:1706.02577v1cs.CV

TL;DR

Behavioral video analysis needs tracking software that is accessible, flexible, robust, and efficient for diverse organisms and large datasets. The paper presents ToxTrac, an open-source automated image-based tracker, and reports accuracy at least comparable to other multiple-animal programs with significantly faster processing. ToxTrac also preserves individual identities during occlusions, although its original design targeted one animal per arena and it has documented Windows and file-naming constraints.

  • Problem

    Existing behavioral-video analysis programs are limited by cost, speed, usability, and flexibility despite the growing use of behavioral assays.

  • Method

    The paper presents ToxTrac, an automated open-source image-based tracking program for multiple organisms, using Kalman tracking, feature extraction, and optional trajectory-fragment linking.

  • Results

    ToxTrac is at least as accurate as other programs for simultaneous multiple-individual tracking and processes data significantly faster; one evaluation identified animals in 99.9% of frames.

  • Takeaways & Limitations

    ToxTrac provides fast tracking for multiple species and arenas while handling occlusions and preserving individual identities.

Abstract

from arXiv · show

1. Behavioral analysis based on video recording is becoming increasingly popular within research fields such as; ecology, medicine, ecotoxicology, and toxicology. However, the programs available to analyze the data, which are; free of cost, user-friendly, versatile, robust, fast and provide reliable statistics for different organisms (invertebrates, vertebrates and mammals) are significantly limited. 2. We present an automated open-source executable software (ToxTrac) for image-based tracking that can simultaneously handle several organisms monitored in a laboratory environment. We compare the performance of ToxTrac with current accessible programs on the web. 3. The main advantages of ToxTrac are: i) no specific knowledge of the geometry of the tracked bodies is needed; ii) processing speed, ToxTrac can operate at a rate >25 frames per second in HD videos using modern desktop computers; iii) simultaneous tracking of multiple organisms in multiple arenas; iv) integrated distortion correction and camera calibration; v) robust against false positives; vi) preservation of individual identification if crossing occurs; vii) useful statistics and heat maps in real scale are exported in: image, text and excel formats. 4. ToxTrac can be used for high speed tracking of insects, fish, rodents or other species, and provides useful locomotor information. We suggest using ToxTrac for future studies of animal behavior independent of research area. Download ToxTrac here: https://toxtrac.sourceforge.io

Introduction

Behavioral video assays are increasingly important, but existing tracking software is often costly, slow, laborious, or difficult to use. ToxTrac is presented as a flexible, open-source program designed for fast, robust tracking across species, arenas, and multiple organisms.

  • Motivation: Behavioral assays can detect effects of environmental substances, but video-based methods generate large datasets that are laborious and time-consuming to analyze.Behavioral changes are used in ecology, animal behavior, ecotoxicology, and medical research.
  • Motivation: Available tracking programs are often costly, slow, programming-dependent, or tied to proprietary software.
  • Requirements: Reliable tracking software should accurately locate organisms, analyze long time series, adapt to new setups, operate efficiently, and process several experiments simultaneously.
  • Existing limitations: Existing approaches have important limitations, including false-positive sensitivity, unreliable identity preservation after occlusions, experimental complexity, and high computational cost.The Kalman filter follows the closest detected object, while fingerprint-based identity tracking can be unsuitable for real-time video.
  • Contribution: ToxTrac is introduced as a free, open-source program that rapidly tracks multiple species and animals across multiple arenas while handling occlusions and preserving individual identities.The software is intended to be flexible, robust over time, and suitable for fast data analysis without requiring specific knowledge of animal geometry.

Performance and results of the program

ToxTrac was evaluated across organisms, arenas, illumination conditions, and comparisons with accessible tracking tools. It showed high detection accuracy, faster tracking, and strong identity preservation, while competing tools had narrower operating limits.

  • Compared tools showed constraints including rigid-body assumptions, shape-separation errors, limited arena or animal counts, resolution restrictions, and instability.
  • ToxTrac and IdTracker achieved average detection rates of 99.2% and 95.9%, respectively, in non-occluded experiments.
  • Detection rates decreased during occlusions, although IdTracker performed better in fish-like multi-animal experiments because of its resegmentation stage.
  • ToxTrac tracking times were significantly lower than IdTracker’s across all tested cases under comparable computer settings.
  • IdTracker could not process the cockroach experiment at its original resolution and performed poorly on high-resolution videos.
  • ToxTrac preserved tracked-animal identity in 99.6% of cases for tracks longer than 50 frames and achieved 92.2% accuracy for shorter tracks.

Discussion

ToxTrac is presented as a broadly applicable behavioral-tracking tool that supports diverse organisms, multiple arenas, and identity preservation during occlusions or crossings. It also provides locomotor and spatial-behavior measures, practical calibration and bulk-processing features, and performance comparable to or better than other programs.

  • The software measures locomotor activity, wall proximity, region-based behavior, and trajectories relevant to exploration, boldness, anxiety, and activity.
  • ToxTrac tracks organisms with different morphologies and movement patterns, including ants, cockroaches, salmon, tadpoles, zebrafish, and guppies.
  • 99.9% of frames contained detected animals in the reported assays, a level considered highly acceptable for common behavioral applications.
  • ToxTrac handles occlusions and multiple animals while preserving individual identities through a fragment linking algorithm.
  • ToxTrac was at least as accurate as other programs, supported multiple-individual tracking, and processed data significantly faster.
  • Calibration corrects image distortion and enables real-scale outputs, while bulk processing supports analyzing hundreds of videos in one step.

Tables

ToxTrac was evaluated across datasets involving seven organisms, with tracking performance compared against IdTracker and identity preservation assessed separately.

  • Table 1 reports tracking performance for eight datasets spanning seven different organisms.
  • 99.99% detection and 00:04:12 tracking time were reported for ToxTrac on the ant dataset.
  • 99.99% detection and 00:12:20 tracking time were reported for ToxTrac on the cockroach dataset.
  • 99.99% detection and 00:00:15 tracking time were reported for ToxTrac on the mice dataset.
  • 99.52% detection and 00:18:33 tracking time were reported for ToxTrac on the salmon dataset.
  • 99.99% detection and 00:00:18 tracking time were reported for ToxTrac on the tadpole dataset.
  • 65,518 detections yielded 99.49% identity preservation, with 0.37% and 0.14% in the other reported categories.

1. Requirements

ToxTrac requires a compatible Windows computer, suitable video quality, controlled illumination and backgrounds, and consistent experimental conditions for reliable tracking.

  • ToxTrac runs on Windows 7 or later and 64-bit hardware, with at least 8 GB RAM and a recommended 2.0+ GHz quad-core processor.
  • The software supports .avi videos with any resolution and framerate, including MPEG-4 and x264 compression, but recommends the highest possible quality.
  • Animal size should be at least 50 pixels, and 25 fps is usually sufficient although faster animals or multi-animal experiments may require higher framerates.
  • Tracking areas should be bright, uniform regions with strong contrast against animals, while dark arena edges and corners should be excluded.
  • Reflections, shadows, and inseparable moving or static objects can make tracking difficult or impossible.
  • Identity tracking is not recommended for more than 10-20 animals in one experiment, depending on occlusion degree.
  • The setup should remain isolated from external interference and unchanged between calibration, experiments, and jointly processed recordings.
  • Diffuse illumination preserves texture and mitigates shadows, whereas backlight highlights shapes but hides object textures.

1. Start Screen

ToxTrac organizes analyses around projects containing compatible video sequences, calibration and configuration data, and produces processed tracking outputs after analysis.

  • The Start Screen supports creating, saving, loading, and merging projects, with merging requiring matching conditions, calibration, and arena parameters.
  • A project uses one calibration and configuration while allowing several video sequences, whose recordings must share experimental conditions and camera parameters.
  • When analysis begins, every sequence is processed and its results are saved in subfolders, then combined for population statistics.
  • Project files store video inputs, configuration, arena definitions, arena names, calibration parameters, and primary tracking outputs.
  • The camera model defines spatial coordinates and distorted coordinates using radial, tangential, and prism distortion coefficients.
  • Calibration can use chessboard images recorded under the experiment’s conditions and resolution, or camera-model parameters entered manually.

3. Calibration screen

ToxTrac calibrates camera geometry and defines arenas and tracking areas from video frames, using corrected grayscale images and selectable automatic or manual procedures.

  • Calibration: Calibration uses image sequences and camera parameters to estimate pose and correct optical distortion.
  • Arena definition: Tracking areas should be uniform, bright, well illuminated regions containing dark, high-contrast objects.
  • Arena definition: Arena selection uses a sample video frame to define tracking areas.
  • Automatic selection: Automatic selection corrects distortion, converts the image to normalized 8-bit grayscale, thresholds bright regions, and applies morphological closing.
  • Manual selection: Manual selection can fit detected areas to minimum enclosing circles and reduce their radii by a chosen pixel value.
  • Background subtraction: Background subtraction classifies pixels as background or foreground using Bayesian probabilities and an adaptive observation window.

2. Detection screen

The detection screen identifies animal objects and supports real-time tracking with a Kalman filter, assignment checks, collision handling, and configurable detection controls.

  • Detection screen: The detection screen lets users navigate frames, set intensity and object-size thresholds, and enable background subtraction.
  • Tracking model: The Kalman filter models two-dimensional motion with position, velocity, and constant acceleration, then updates estimates from noisy measurements.
  • Track assignment: The filter assigns detections to tracks using predicted-position distances and the Hungarian optimization algorithm.
  • Identity features: Identity features comprise intensity histograms and Intensity and Contrast Center Maps for detected bodies.
  • Assignment validation: Additional position and size-change checks reduce the chance of assigning an incorrect detection to a track.
  • Collision handling: When individuals overlap or cross, conflicted tracks are marked inactive and new tracks are generated.

2. Fragment Identification

Fragment identification is an optional post-processing step that reconnects trajectory fragments to preserve individual identities after occlusion.

  • Fragment identification: Fragment identification compares stored trajectory features to determine which tracks belong to the same animal.
  • Similarity calculation: Similarity calculations can be restricted to similarly sized samples with minimum histogram correlation.
  • Track assignment: A Hungarian-algorithm variant assigns groups of long tracks, after which remaining tracks are assigned iteratively by best correlation.
  • Iterative assignment: Each assignment updates the similarity matrix and reduces uncertainty for the remaining tracks.
  • Memory control: The number of active and inactive tracks retained in memory is user-configurable because fragment identification requires substantial memory.

3. Tracking screen

The tracking and statistics screens organize multi-animal tracking and expose individual- and population-level locomotor measures.

  • Tracking screen: The tracking screen supports selecting animal counts and choosing among fragment-identification options.
  • Individual statistics: Individual statistics can be viewed for selected arenas and sequences with configurable mobility and frozen-event thresholds.
  • Population statistics: Population statistics display tracking measures for the entire population.

2. Graphical outputs screen

The graphical outputs present tracking results superimposed on arena images in real scale, using color-coded heat maps and configurable arena orientations.

  • Graphical outputs are shown superimposed on the arena picture and in real scale.
  • Heat maps encode normalized zone-use frequency with a linear color scale.The scale is illustrated in Figure 19.
  • Available outputs include a plain arena view and heat maps based on distance to arena edges.The listed edge options include North and West Edge outputs.
  • Arena orientations can be changed in the projected virtual arena for the entire population.Options include same orientation and vertical or horizontal mirroring.

3. Output Files

ToxTrac exports tracking, real-space, locomotor, edge-use, and exploration results for individuals, populations, arenas, and sequences in structured files and graphical formats.

  • The project automatically creates folders and files for configuration, main individual and population statistics, population results, and arena- or sequence-specific results.Creation of most files can be enabled or disabled.
  • Tracking_RealSpace files translate detected positions into calibrated real-space coordinates, typically millimeters, with time in seconds.Arena-specific coordinates use arena-dependent reference points, while population coordinates can be projected to a common virtual arena.
  • Trajectory outputs display colored animal tracks over arena images in real scale, with different tracks in one arena shown in different colors.Examples include five fish in one arena and three fish across four arenas.
  • Instantaneous speed and acceleration are derived from Tracking_RealSpace data using a user-defined sampling distance.Acceleration values represent accelerations and decelerations as positive values.
  • Edge-distance and exploration analyses report frame counts, time based on frame rate, and detection frequency for spatial zones.Edge analyses use user-defined distance areas, while exploration divides arenas into user-sized square zones.
  • The output matrix may exceed the actual arena because its size is predefined for memory reasons.Unused cells are filled with zeros.
  • Heat-map outputs represent normalized zone-use frequency, with colors progressing from blue through green and yellow to red as frequency increases.Exploration and edge-use graphics are superimposed on the arena image.

1. Graphic parameters (ColorIni.txt)

ColorIni.txt stores graphical parameters controlling fonts, line widths, and colors, with values scaled to image or window resolution and represented in BGR order.

  • ColorIni.txt contains general graphical parameters and color parameters.
  • Graphical parameters control cosmetic features such as font sizes, line widths, and colors.These settings do not change the underlying analysis.
  • Width and size values are scaled according to image or window resolution.
  • Color values use RGB values in inverse BGR order, consistent with OpenCV representation.

2. Configuration parameters (Configuration.txt)

Configuration.txt organizes project settings into parameter groups covering calibration, arena definition, detection, tracking, identity, outputs, and data analysis.

  • Configuration.txt stores project configuration parameters and loads defaults from ConfigurationIni.txt when a project starts or is created.Parameter groups are identified by unique codes and documented across Tables 29–59.
  • Calibration and arena-definition settings include distortion modeling, arena orientation, polygonal tracking-area approximation, morphology, minimum arena area, and optional ellipse fitting.Several settings are available through the interface, while some apply only to automatic or manual selection.
  • Detection and preprocessing settings include erosion operations, background and detection parameters, opening and closing, dilation, erosion, filtering, and Kalman-filter configuration.The configuration tables separately document these parameter groups.
  • Tracking and identity settings cover Kalman-filter acceptance and deletion conditions, track counts, identity algorithms, feature selection, collision handling, and track fusion.
  • Output and analysis settings control minimum track size, correlation and maximum-track parameters, output generation, arena and zone analysis, speed, frozen events, transitions, post-processing, and other analyses.Main video parameters are also included.
  • Arena orientation uses codes for same orientation, horizontal mirror, vertical mirror, and combined vertical and horizontal mirroring.
Loading 1706.02577v1…