Imagine watching someone put objects on shelves without being told which movements to make. Learning from that video alone would be difficult for a robot too. A research team led by Jun Cen investigated whether videos could help robots perform tasks that were missing from their usual training examples.
The researchers used learning from video. One part of the system predicts what the next moments of an activity might look like. Another part chooses movements that could produce those pictures. This is not the same as copying a person directly. A robot has different hands, proportions and limits, so its own movements must still be calculated.
In a computer test of unfamiliar tasks, the starting system succeeded 25.07% of the time. Adding videos of those tasks, without recorded movement commands, increased success to 55.90%. Providing just one example with commands for each new task raised it to 70.70%. Those numbers describe a particular test collection, not every robot in every room.
WorldBridge[1] combines separate video-prediction and action-generation experts, connected by structured attention and trained using joint flow matching. Its visual memory operates on two temporal scales. The authors distinguish video-only coverage of unfamiliar tasks from adding one action-labelled demonstration per task. The latter raises success to 70.70%, but requires a different supervision budget. Treating these two results as though they used identical training data would obscure the contribution of the action labels.
The experiment offers a reason to collect useful videos as well as demonstrations with movement commands. It does not mean a household robot can watch one clip and immediately cook dinner or assemble furniture. The researchers tested specific tasks under controlled conditions.
The intriguing distinction is between seeing what should happen and knowing how to make it happen. Video shows objects changing position but contains no joint commands or forces. The model tries to recover the missing connection between the picture and the robot's action. One example with commands raises success to 70.70%, showing the value of even a small amount of physically grounded data. A plausible-looking video prediction, however, may still describe a movement a particular robot cannot perform.
The reported 96.23% on the full RoboTwin 2.0[2] setting belongs to a different protocol from the unfamiliar-task experiment. Transfer to another gripper, camera arrangement or physical robot is not established by these figures. A reproducible comparison must specify exactly which task videos and action labels were available.
The paper also reports 96.23% in a separate full-data evaluation, not on the same unfamiliar tasks. Whether this approach can learn directly from ordinary household videos remains unresolved: casual recordings rarely include precise robot movement labels.