T3 WALKTHROUGH
Object tracking¶
Takes RGB frames in order, with sequence boundaries and an initialization, and returns boxes with persistent track IDs.
Scene 049, Interaction, frame 00185. The overlay shows the released ground-truth masks, whose identities persist across the sequence.
Connect your model¶
Put your prediction function in my_model.py. Load the model or API client once, at module level, rather than on every call.
# tracker is yours: initialize it once per sequence, then update it per frame.
def predict_tracks(rgb):
return tracker.update(rgb) # [{"track_id": "1", "boxes": boxes_xyxy}, ...]
Keep state between calls, reset it at each sequence boundary, and keep track IDs stable. Paper HOTA and IDF1 numbers come from the sequence evaluator described below.
Run the benchmark¶
Install the toolkit with pip install 'rpx-benchmark[hub,schemas]'. The smoke test checks the install; the second tab scores your model.
This runs the real evaluator on a few synthetic inputs with a placeholder model. Nothing is downloaded and no weights are needed. The output directory must be new or empty.
Run this from the directory that contains my_model.py. Without --manifest, the command downloads the data it needs first.
python -m rpx_benchmark.examples.benchmark_tasks \
--task T3 \
--model my_model:predict_tracks \
--model-name my-model \
--model-revision exact-checkpoint-or-api-version \
--output results/my-model/T3 \
--device cuda \
--split easy \
--max-samples 10
Ten samples are enough to check the wiring; drop --max-samples to score the whole split. To use data you already have on disk, add --manifest path/to/manifest.json (see using local data).
Read the results¶
The output directory holds result.json, summary.md, per-sample tables and cell logs. run_config.json records the model revision and the manifest hash. From Python, the run_* functions return (result, report, paths).
Check that the number of scored samples matches what you asked for. Keep the per-sample records: Φ and 𝒥min are computed from them. Latency and memory are measured separately, with the hardware profiler.
Scoring and protocol¶
This command runs the per-frame tracking adapter and the registered tracking metrics. It is a way to check your integration, not the sequence-level evaluator used in the paper. Your module owns the tracker state and must reset it between sequences; a detector that treats every frame independently isn't a tracker.
For sequence-level metrics and the paper's trackers, use run_tracking_paper.py and run_tracking_gate.py with the pinned evaluator; their --help lists the per-model initialization, mask outputs and sequence settings. Don't report per-frame scores from this command as HOTA or IDF1.
Before a full run
Check units, shapes, input order and sample IDs on a small run first. Pin the model revision, and keep the split and preprocessing the same for every model you compare. The smoke test only exercises the evaluator; your model still needs its own environment and weights.
Workflow figure
T3 workflow (SVG): the inputs, adapter and scoring stages for this task.
