T2 WALKTHROUGH
Video depth¶
Takes an ordered T × H × W × 3 RGB clip from one phase and returns a T × H × W depth sequence.
Scene 049, Interaction, frames 00185 and 00186. Real clips are longer; their length comes from the manifest.
Connect your model¶
Put your prediction function in my_model.py. Load the model or API client once, at module level, rather than on every call.
import numpy as np
def predict_video_depth(rgb):
return np.asarray(model.predict_clip(rgb), dtype=np.float32)
Return one depth map per input frame, in the order you received them. A constant map is rejected as degenerate.
Run the benchmark¶
Install the toolkit with pip install 'rpx-benchmark[hub,schemas]'. The smoke test checks the install; the second tab scores your model.
This runs the real evaluator on a few synthetic inputs with a placeholder model. Nothing is downloaded and no weights are needed. The output directory must be new or empty.
Run this from the directory that contains my_model.py. Without --manifest, the command downloads the data it needs first.
python -m rpx_benchmark.examples.benchmark_tasks \
--task T2 \
--model my_model:predict_video_depth \
--model-name my-model \
--model-revision exact-checkpoint-or-api-version \
--output results/my-model/T2 \
--device cuda \
--split easy \
--max-samples 10
Ten samples are enough to check the wiring; drop --max-samples to score the whole split. To use data you already have on disk, add --manifest path/to/manifest.json (see using local data).
Read the results¶
The output directory holds result.json, summary.md, per-sample tables and cell logs. run_config.json records the model revision and the manifest hash. From Python, the run_* functions return (result, report, paths).
Check that the number of scored samples matches what you asked for. Keep the per-sample records: Φ and 𝒥min are computed from them. Latency and memory are measured separately, with the hardware profiler.
Scoring and protocol¶
Your function must keep the input frame order and return the same T. Pass --depth-output-kind relative only for up-to-scale predictions; those get a per-clip scale and shift fit before scoring. Compare models with the same clip length, frame sampling and alignment.
Scores combine per-frame depth accuracy with two temporal metrics, tgm and tgse. The paper's metric vector leaves out the pose- and flow-dependent diagnostics. Here --max-samples counts clips, not frames. VideoDepthRunConfig has frame-budget and sampling options for ablations.
Before a full run
Check units, shapes, input order and sample IDs on a small run first. Pin the model revision, and keep the split and preprocessing the same for every model you compare. The smoke test only exercises the evaluator; your model still needs its own environment and weights.
Workflow figure
T2 workflow (SVG): the inputs, adapter and scoring stages for this task.
