Introduction

Trackforge is a unified, high-performance multi-object tracking library written in Rust and exposed to Python via PyO3. It implements four production-ready tracking algorithms on top of a shared Kalman filter, so you can swap trackers without changing your integration code.

It is designed as the CPU “glue” between a GPU object detector and your application: you pass in detection boxes each frame and get back stable track identities.

Trackers at a glance

Every tracker accepts the same detection tuple, ([x, y, w, h], score, class_id), and ships for both Python and Rust.

This book is a narrative guide. For the full API surface, see the Rust API on docs.rs and the Python API reference.

Getting Started

Install

Rust

[dependencies]
trackforge = "0.3"

Python

pip install trackforge

Your first tracker

The detection format is the same everywhere: a list of ([x, y, w, h], score, class_id) tuples, where x, y is the top-left corner in pixels.

Rust

#![allow(unused)]
fn main() {
use trackforge::trackers::byte_track::ByteTrack;

let mut tracker = ByteTrack::new(0.5, 30, 0.8, 0.6);

let detections = vec![
    ([100.0, 100.0, 50.0, 100.0], 0.9, 0),
    ([200.0, 200.0, 60.0, 120.0], 0.85, 0),
];

let tracks = tracker.update(detections);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Python

from trackforge import BYTETRACK

tracker = BYTETRACK(track_thresh=0.5, track_buffer=30, match_thresh=0.8, det_thresh=0.6)

detections = [
    ([100.0, 100.0, 50.0, 100.0], 0.9, 0),
    ([200.0, 200.0, 60.0, 120.0], 0.85, 0),
]

for track_id, tlwh, score, class_id in tracker.update(detections):
    print(f"ID: {track_id}, Box: {tlwh}")

Call update once per frame. Each tracker keeps its own state and returns the confirmed tracks for the current frame.

Trackers

All trackers share the same Kalman filter and the same detection input, and differ only in how they associate detections to existing tracks. Pick based on your scene and whether you have a Re-ID model.

The Kalman filter uses an 8-dimensional state [x, y, a, h, vx, vy, va, vh], where (x, y) is the box centre, a is the aspect ratio, and h is the height. Detections in [x, y, w, h] (top-left) form are converted to and from this representation internally.

SORT

Simple Online and Realtime Tracking (arXiv 1602.00763). Pairs a Kalman filter with greedy IoU matching. Built for speed; best when objects rarely overlap.

#![allow(unused)]
fn main() {
use trackforge::trackers::sort::Sort;

let mut tracker = Sort::new(1, 3, 0.3);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: raise max_age to bridge short occlusions (at the cost of more false tracks); lower iou_threshold for densely packed objects; raise min_hits to suppress false starts from a noisy detector.

ByteTrack

ByteTrack (arXiv 2110.06864). A two-stage IoU tracker that associates every detection, not just high-confidence ones, to recover objects that are briefly occluded or partially off-screen. A strong recall improvement over SORT at minimal extra cost.

#![allow(unused)]
fn main() {
use trackforge::trackers::byte_track::ByteTrack;

let mut tracker = ByteTrack::new(0.5, 30, 0.8, 0.6);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: lower track_thresh (~0.3) to feed more low-confidence detections into stage two; raise track_buffer (~60) when the detector drops frames; lower match_thresh (~0.7) for fast-moving objects where IoU falls off quickly.

OC-SORT

Observation-Centric SORT (arXiv 2203.14360, CVPR 2023). Extends SORT with three observation-centric mechanisms that reduce drift during occlusions:

• OCV computes velocity from consecutive detections rather than the Kalman state.
• OCM adds a direction-consistency bonus before matching: a candidate whose direction from the track’s last observation aligns with the track’s velocity gets a higher effective IoU.
• ORU corrects the Kalman filter after re-association by replaying interpolated observations between the last seen position and the current detection.

No appearance features required, making it a strong upgrade over SORT when occlusions are common but Re-ID is unavailable.

#![allow(unused)]
fn main() {
use trackforge::trackers::ocsort::OcSort;

let mut tracker = OcSort::new(30, 3, 0.3, 3, 0.2);
let tracks = tracker.update(vec![([100.0, 100.0, 50.0, 100.0], 0.9, 0)]);
for t in tracks {
    println!("ID: {}, Box: {:?}", t.track_id, t.tlwh);
}
}

Parameters

Tuning: raise max_age for long occlusions; raise delta_t for smoother velocity at the cost of responsiveness; raise inertia toward 1.0 for near-constant-velocity motion, lower it for erratic motion.

DeepSORT

DeepSORT (arXiv 1703.07402). Extends SORT with a Re-ID appearance metric. Confirmed tracks are matched first by cosine distance on appearance embeddings (with Mahalanobis gating), then any remaining tracks fall back to IoU matching. Best for long-term identity maintenance.

Unlike the other trackers, DeepSORT needs an appearance embedding per detection. In Rust you supply an AppearanceExtractor; in Python you pass embeddings directly.

Rust: implement an extractor

use trackforge::traits::AppearanceExtractor;
use trackforge::types::BoundingBox;
use image::DynamicImage;

struct MyExtractor;

impl AppearanceExtractor for MyExtractor {
    fn extract(
        &mut self,
        image: &DynamicImage,
        boxes: &[BoundingBox],
    ) -> Result<Vec<Vec<f32>>, Box<dyn std::error::Error>> {
        // Crop each box, run your Re-ID model, return one embedding per box.
        Ok(boxes.iter().map(|_| vec![0.0_f32; 128]).collect())
    }
}

use trackforge::trackers::deepsort::DeepSort;
let mut tracker = DeepSort::new(MyExtractor, 70, 3, 0.7, 0.2, 100);

Python: bring your own embeddings

from trackforge import DEEPSORT

tracker = DEEPSORT(max_age=70, n_init=3, max_iou_distance=0.7, max_cosine_distance=0.2, nn_budget=100)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
embeddings = [[0.1] * 128]  # one vector per detection, from your Re-ID model

for track_id, tlwh, score, class_id in tracker.update(detections, embeddings):
    print(track_id, tlwh)

Parameters

Tuning: lower max_cosine_distance (~0.15) for stricter Re-ID; raise nn_budget if appearance drifts over time; lower n_init to 1 when detections are reliable and you need tracks immediately.

Deep OC-SORT

Deep OC-SORT (arXiv 2302.11813, ICIP 2023). Extends OC-SORT with appearance, blending a Re-ID embedding cost into the motion association:

• OCM adds a velocity direction-consistency bonus to the IoU before matching.
• ORU replays interpolated observations to correct the Kalman filter after a track is re-associated following a gap.
• Appearance adds a cosine distance to each track’s feature gallery. The appearance weight scales with detector confidence (dynamic appearance) and is gated by max_cosine_distance.
• Camera motion compensation warps track predictions by a caller-supplied affine transform before association, for moving-camera footage.

With appearance_weight = 0 the association reduces to plain OC-SORT, so the appearance term is a strict add-on. This is a clean-room implementation. CMC is applied from a transform you supply (for example estimated with OpenCV); the tracker does not estimate camera motion itself, which keeps the core free of heavy dependencies. Pass the affine as [a, b, tx, c, d, ty] to update.

from trackforge import DEEPOCSORT

tracker = DEEPOCSORT(max_age=30, min_hits=3, iou_threshold=0.3, delta_t=3, inertia=0.2,
                     appearance_weight=0.5, max_cosine_distance=0.2, nn_budget=100)

detections = [([100.0, 100.0, 50.0, 100.0], 0.9, 0)]
embeddings = [[0.1, 0.2, 0.3]]  # one appearance vector per detection
tracks = tracker.update(detections, embeddings)
for track_id, tlwh, score, class_id in tracks:
    print(f"ID: {track_id}, Box: {tlwh}")

Parameters

Tuning: raise appearance_weight when the Re-ID model is reliable and identities matter; lower it (toward 0) to fall back to OC-SORT motion. Tighten max_cosine_distance to only trust strong appearance matches. The motion parameters behave as in OC-SORT.

Citation

@inproceedings{maggiolino2023deepocsort,
  title={Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification},
  author={Maggiolino, Gerard and Ahmad, Adnan and Cao, Jinkun and Kitani, Kris},
  booktitle={IEEE International Conference on Image Processing (ICIP)},
  year={2023}
}

Parameters

The parameters are identical across Python and Rust. In Python they are keyword arguments to the tracker class; in Rust they are positional arguments to Tracker::new(...). Each tracker’s chapter documents its own parameters and tuning advice:

• SORT parameters
• ByteTrack parameters
• OC-SORT parameters
• DeepSORT parameters

The same tables are also published on the documentation site.

Examples

Runnable demos live under examples/ in the repository, with a Python and a Rust entry per tracker.

# Python
python examples/python/byte_track_demo.py

# Rust
cargo run --example byte_track_demo
cargo run --example deepsort_simple
cargo run --example deepsort_ort --features advanced_examples

The Python demos use the usual detector stacks (ultralytics, transformers + torch, torch + torchvision); install what a given demo imports. The deepsort_ort Rust demo needs the advanced_examples feature (ONNX Runtime + OpenCV).