Introduction

Trackforge is a unified, high-performance multi-object tracking library written in Rust and exposed to Python via PyO3. It implements seven production-ready tracking algorithms on top of a shared Kalman filter, so you can swap trackers without changing your integration code.

It is designed as the CPU “glue” between a GPU object detector and your application: you pass in detection boxes each frame and get back stable track identities.

Trackers at a glance

Every tracker accepts the same detection tuple, ([x, y, w, h], score, class_id), and ships for both Python and Rust.

This book is a narrative guide. For the full API surface, see the Rust API on docs.rs and the Python API reference.

Getting Started

Install

Rust

[dependencies]
trackforge = "0.4"

The default build is light and pulls no image codecs. Every tracker runs on detections you pass in, and the appearance trackers run on embeddings you pass in. To have the library produce embeddings for you by running a model over a frame, enable the reid-model feature, which adds the AppearanceExtractor trait and the DeepSort and DeepOcSort wrappers along with the image crate.

[dependencies]
trackforge = { version = "0.3", features = ["reid-model"] }

Python

pip install trackforge

Your first tracker

The detection format is the same everywhere: a list of ([x, y, w, h], score, class_id) tuples, where x, y is the top-left corner in pixels.

Rust

#![allow(unused)]
fn main() {
use trackforge::trackers::byte_track::ByteTrack;

let mut tracker = ByteTrack::new(0.5, 30, 0.8, 0.6);

let detections = vec![
    ([100.0, 100.0, 50.0, 100.0], 0.9, 0),
    ([200.0, 200.0, 60.0, 120.0], 0.85, 0),
];

let tracks = tracker.update(detections);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Python

from trackforge import BYTETRACK

tracker = BYTETRACK(track_thresh=0.5, track_buffer=30, match_thresh=0.8, det_thresh=0.6)

detections = [
    ([100.0, 100.0, 50.0, 100.0], 0.9, 0),
    ([200.0, 200.0, 60.0, 120.0], 0.85, 0),
]

for track_id, tlwh, score, class_id in tracker.update(detections):
    print(f"ID: {track_id}, Box: {tlwh}")

Call update once per frame. Each tracker keeps its own state and returns the confirmed tracks for the current frame.

Trackers

All trackers share the same Kalman filter and the same detection input, and differ only in how they associate detections to existing tracks. Pick based on your scene and whether you have a Re-ID model.

The Kalman filter uses an 8-dimensional state [x, y, a, h, vx, vy, va, vh], where (x, y) is the box centre, a is the aspect ratio, and h is the height. Detections in [x, y, w, h] (top-left) form are converted to and from this representation internally.

SORT

Simple Online and Realtime Tracking (arXiv 1602.00763). Pairs a Kalman filter with greedy IoU matching. Built for speed; best when objects rarely overlap.

#![allow(unused)]
fn main() {
use trackforge::trackers::sort::Sort;

let mut tracker = Sort::new(1, 3, 0.3);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: raise max_age to bridge short occlusions (at the cost of more false tracks); lower iou_threshold for densely packed objects; raise min_hits to suppress false starts from a noisy detector.

ByteTrack

ByteTrack (arXiv 2110.06864). A two-stage IoU tracker that associates every detection, not just high-confidence ones, to recover objects that are briefly occluded or partially off-screen. A strong recall improvement over SORT at minimal extra cost.

#![allow(unused)]
fn main() {
use trackforge::trackers::byte_track::ByteTrack;

let mut tracker = ByteTrack::new(0.5, 30, 0.8, 0.6);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: lower track_thresh (~0.3) to feed more low-confidence detections into stage two; raise track_buffer (~60) when the detector drops frames; lower match_thresh (~0.7) for fast-moving objects where IoU falls off quickly.

OC-SORT

Observation-Centric SORT (arXiv 2203.14360, CVPR 2023). Extends SORT with three observation-centric mechanisms that reduce drift during occlusions:

• OCV computes velocity from consecutive detections rather than the Kalman state.
• OCM adds a direction-consistency bonus before matching: a candidate whose direction from the track’s last observation aligns with the track’s velocity gets a higher effective IoU.
• ORU corrects the Kalman filter after re-association by replaying interpolated observations between the last seen position and the current detection.

No appearance features required, making it a strong upgrade over SORT when occlusions are common but Re-ID is unavailable.

#![allow(unused)]
fn main() {
use trackforge::trackers::ocsort::OcSort;

let mut tracker = OcSort::new(30, 3, 0.3, 3, 0.2);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: raise max_age for long occlusions; raise delta_t for smoother velocity at the cost of responsiveness; raise inertia toward 1.0 for near-constant-velocity motion, lower it for erratic motion.

DeepSORT

DeepSORT (arXiv 1703.07402). Extends SORT with a Re-ID appearance metric. Confirmed tracks are matched first by cosine distance on appearance embeddings (with Mahalanobis gating), then any remaining tracks fall back to IoU matching. Best for long-term identity maintenance.

Unlike the other trackers, DeepSORT needs an appearance embedding per detection. In Rust you supply an AppearanceExtractor; in Python you pass embeddings directly.

The AppearanceExtractor trait and the DeepSort wrapper below live behind the reid-model feature, which adds the image crate. Enable it with features = ["reid-model"]. If you already have embeddings, skip the extractor and drive DeepSortTracker directly on the default build.

Rust: implement an extractor

use trackforge::traits::AppearanceExtractor;
use trackforge::types::BoundingBox;
use image::DynamicImage;

struct MyExtractor;

impl AppearanceExtractor for MyExtractor {
    fn extract(
        &mut self,
        image: &DynamicImage,
        boxes: &[BoundingBox],
    ) -> Result<Vec<Vec<f32>>, Box<dyn std::error::Error>> {
        // Crop each box, run your Re-ID model, return one embedding per box.
        Ok(boxes.iter().map(|_| vec![0.0_f32; 128]).collect())
    }
}

use trackforge::trackers::deepsort::DeepSort;
let mut tracker = DeepSort::new(MyExtractor, 70, 3, 0.7, 0.2, 100);

Python: bring your own embeddings

from trackforge import DEEPSORT

tracker = DEEPSORT(max_age=70, n_init=3, max_iou_distance=0.7, max_cosine_distance=0.2, nn_budget=100)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
embeddings = [[0.1] * 128]  # one vector per detection, from your Re-ID model

for track_id, tlwh, score, class_id, det_ind in tracker.update(detections, embeddings):
    print(track_id, tlwh)

Parameters

Tuning: lower max_cosine_distance (~0.15) for stricter Re-ID; raise nn_budget if appearance drifts over time; lower n_init to 1 when detections are reliable and you need tracks immediately.

Deep OC-SORT

Deep OC-SORT (arXiv 2302.11813, ICIP 2023). Extends OC-SORT with appearance, blending a Re-ID embedding cost into the motion association:

• OCM adds a velocity direction-consistency bonus to the IoU before matching.
• ORU replays interpolated observations to correct the Kalman filter after a track is re-associated following a gap.
• Appearance adds a cosine distance to each track’s feature gallery. The appearance weight scales with detector confidence (dynamic appearance) and is gated by max_cosine_distance.
• Camera motion compensation warps track predictions by a caller-supplied affine transform before association, for moving-camera footage.

With appearance_weight = 0 the association reduces to plain OC-SORT, so the appearance term is a strict add-on. This is a clean-room implementation. CMC is applied from a transform you supply (for example estimated with OpenCV); the tracker does not estimate camera motion itself, which keeps the core free of heavy dependencies. Pass the affine as [a, b, tx, c, d, ty] to update.

from trackforge import DEEPOCSORT

tracker = DEEPOCSORT(max_age=30, min_hits=3, iou_threshold=0.3, delta_t=3, inertia=0.2,
                     appearance_weight=0.5, max_cosine_distance=0.2, nn_budget=100)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
embeddings = [[0.1, 0.2, 0.3]]  # one appearance vector per detection
tracks = tracker.update(detections, embeddings)
for track_id, tlwh, score, class_id, det_ind in tracks:
    print(f"ID: {track_id}, Box: {tlwh}")

Parameters

Tuning: raise appearance_weight when the Re-ID model is reliable and identities matter; lower it (toward 0) to fall back to OC-SORT motion. Tighten max_cosine_distance to only trust strong appearance matches. The motion parameters behave as in OC-SORT.

Citation

@inproceedings{maggiolino2023deepocsort,
  title={Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification},
  author={Maggiolino, Gerard and Ahmad, Adnan and Cao, Jinkun and Kitani, Kris},
  booktitle={IEEE International Conference on Image Processing (ICIP)},
  year={2023}
}

BoT-SORT

BoT-SORT (arXiv 2206.14651). Extends ByteTrack’s two-stage cascade with two additions:

• Camera motion compensation warps each track’s Kalman prediction by a caller-supplied affine transform before association, so tracking survives panning and zooming cameras.
• Appearance fusion combines a cosine distance to each track’s appearance embedding with the IoU distance in the high-confidence stage. Appearance is used only when it is confident (below appearance_thresh) and the pair is spatially close (IoU distance below proximity_thresh); the fused cost is the smaller of the two. Each track keeps an exponential moving average of its embeddings.

With no embeddings the association reduces to ByteTrack with camera motion, so appearance is a strict add-on. This is a clean-room implementation. CMC is applied from a transform you supply (for example estimated with OpenCV); the tracker does not estimate camera motion itself. Pass the affine as [a, b, tx, c, d, ty] to update.

from trackforge import BOTSORT

tracker = BOTSORT(track_thresh=0.5, track_buffer=30, match_thresh=0.8, det_thresh=0.6,
                  proximity_thresh=0.5, appearance_thresh=0.25)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
embeddings = [[0.1, 0.2, 0.3]]  # one appearance vector per detection; omit for motion only
tracks = tracker.update(detections, embeddings)
for track_id, tlwh, score, class_id, det_ind in tracks:
    print(f"ID: {track_id}, Box: {tlwh}")

Parameters

Tuning: supply a camera-motion affine on moving-camera footage; leave it out for a static camera. Provide embeddings when a Re-ID model is available and identities matter, and tighten appearance_thresh to only trust strong appearance matches. The two-stage thresholds behave as in ByteTrack.

Citation

@article{aharon2022botsort,
  title={BoT-SORT: Robust Associations Multi-Pedestrian Tracking},
  author={Aharon, Nir and Orfaig, Roy and Bobrovsky, Ben-Zion},
  journal={arXiv preprint arXiv:2206.14651},
  year={2022}
}

TrackTrack

TrackTrack (CVPR 2025). A track-centric online tracker built on a ByteTrack-style two-stage lifecycle, with two contributions.

• Track-perspective association. Rather than solving one global assignment, each track picks its own best detection and a pair matches only when the choice is mutual. The loop repeats with a cost gate that tightens each round. High and low confidence detections share one pass; low ones carry a penalty instead of running as a separate stage. The cost fuses a height-modulated IoU, an optional appearance term, a confidence projection, and a velocity-direction term.
• Track-aware initialization. A leftover detection starts a new track only if it clears an init threshold and does not overlap an existing active track, or a more confident leftover, by more than tai_thresh.

Appearance is optional. Pass embeddings to use the Re-ID term, or an empty slice to track on motion only.

This port keeps the two contributions and the fused cost on the shared 8-dimensional Kalman filter. It uses a simplified velocity-direction term and does not reproduce the paper’s detector-level NMS recovery pool, which needs access to the detector’s suppressed boxes.

Parameters

Python

from trackforge import TRACKTRACK

tracker = TRACKTRACK(det_thresh=0.6, match_thresh=0.7, track_buffer=30, min_hits=3)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
tracks = tracker.update(detections)
for track_id, tlwh, score, class_id, det_ind in tracks:
    print(f"ID={track_id}  box={tlwh}")

Rust

use trackforge::trackers::tracktrack::TrackTrack;

let mut tracker = TrackTrack::new();

let detections = vec![([100.0_f32, 100.0, 50.0, 100.0], 0.9_f32, 0_i64)];
let tracks = tracker.update(detections, &[]);
for t in &tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}

Parameters

The parameters are identical across Python and Rust. In Python they are keyword arguments to the tracker class. In Rust they are positional arguments to Tracker::new(...), or fields on a params struct passed to Tracker::from_params(...).

Two settings recur in almost every tracker. How many frames a lost track survives before it is dropped, called max_age in most trackers and track_buffer in ByteTrack and BoT-SORT. And how many matched frames in a row confirm a new track, called min_hits in most and n_init in DeepSORT. In Rust these two live in CommonParams, which each tracker’s params struct embeds as common.

IoU thresholds come in two opposite flavors. A minimum IoU (iou_threshold, higher is stricter) in SORT, OC-SORT, and Deep OC-SORT. And a maximum IoU distance of one minus IoU (match_thresh, max_iou_distance, lower is stricter) in ByteTrack, BoT-SORT, and DeepSORT. The full table with a plain description of every parameter is on the documentation site.

Each tracker’s chapter documents its own parameters and tuning advice:

• SORT parameters
• ByteTrack parameters
• OC-SORT parameters
• DeepSORT parameters
• Deep OC-SORT parameters
• BoT-SORT parameters
• TrackTrack parameters

The same tables are also published on the documentation site.

Examples

Runnable demos live under examples/ in the repository, with a Python and a Rust entry per tracker.

# Python
python examples/python/byte_track_demo.py

# Rust
cargo run --example byte_track_demo
cargo run --example deepsort_simple --features reid-model
cargo run --example deepsort_ort --features advanced_examples
cargo run --example det_ind_demo

The Python demos use the usual detector stacks (ultralytics, transformers + torch, torch + torchvision); install what a given demo imports. The deepsort_simple Rust demo needs the reid-model feature and the deepsort_ort demo needs the advanced_examples feature (ONNX Runtime + OpenCV).