🎬 TransVLM — Detect Any Shot Transition

Hard cuts are easy. Dissolves, fades, wipes and whip-pans are not — they are gradual, and classic shot-boundary detectors either miss them or fire on camera motion. TransVLM fuses optical flow into the vision tower at the input stage (a 6-channel patch embed over RGB ⊕ flow) so a Qwen3-VL backbone can see motion as well as appearance, and asks it to write out the start and end time of every transition it finds.

This Space runs the authors' full pipeline: 25 fps resample → smart_resize → whole-video NeuFlow v2 optical flow → 10 s sliding windows → JSON parse → merge. model · code · paper
4 30
4 12
2 12
0 0.5
128 2048

The model was trained and evaluated at 25 fps, a 524,288-pixel per-frame budget and 10 s / 9 s windows. Changing these changes the signal the model sees.

Detected transitions

Examples — the authors' own STD-Synth showcase clips
Video Seconds to analyse

Ground truth published with the paper, for the three examples above:

clip ground-truth transitions
abrupt_cuts three hard cuts at 5.10 s, 7.26 s, 12.50 s
gradual_transitions 2.52–8.20 s and 9.12–9.92 s
long_dissolve 2.35–10.00 s (7.6 s squeeze) and 11.98–12.36 s