ONE POLICY · TWO HANDS

DONIC

Generalist Dexterous Bimanual Tracking Control
from Human Demonstrations

sequences: 1,575
objects: 229
interaction: 14.77 h
cameras: 32
unseen-traj SR: 86.79%
unseen-obj SR: 57.32%

A NOTE

Simulation rollouts. Every rollout on this page is a DONIC policy execution in Isaac Gym. The unseen-object cases track references whose objects were held out from training, without fine-tuning.

Same policy, both hands. A single network controls the left and right hand together; the reference decides when each hand engages the object.

Gallery

One checkpoint. Many bimanual interactions.

Tags mark where a reference comes from: unseen object clips involve objects held out from training, BiHOI clips come from our multi-view captures, and comparison clips reappear side by side below. Hover to pause.

BiHOI dataset

1,575 bimanual sequences. Captured from 32 views.

32 calibrated RGB cameras surround a tabletop workspace and record at 1920 × 1232 pixels and 30 fps. We reconstruct both hands and the manipulated object, then retarget the motion to robot hands.

Both hands in contact. Share of contact frames in which both hands touch the object, from Table 1 of the paper.

Generalization

New objects. No fine-tuning.

Held-out objects from public datasets, from the human recording to the learned bimanual execution. Shorter source clips hold their final frame.

Method

One shared policy. Verified demonstrations in the loop.

Object-centered per-hand reward

Object tracking stays active throughout the motion, while each hand is supervised by the contacts its reference demands, so one hand's progress is not masked by the other.

Clip-Centered Advantages

Subtracting each clip's mean estimated advantage before the PPO update removes reference-dependent offsets while keeping the differences within a clip.

PPO + behavior cloning

Successful executions are replay-verified and recycled into a demonstration library that supplies behavior-cloning targets in later updates.

Side by side

Follow the object. Coordinate both hands.

DONIC against bimanual adaptations of DexTrack and Dexplore, trained on the same data pool in the same simulator. One transport drives all four views.

Dexplore-bi and DONIC share the reference conversion; DexTrack-bi uses its own converted reference and scene, so camera views are not spatially aligned. Playback loops at the longest clip.

Results

One checkpoint. Both held-out splits.

Method	T_o ↓cm	R_o ↓rad	T_h ↓cm	R_h ↓rad	E_finger ↓rad	SR ↑%	Comp ↑%

Table 2 of the paper. Metrics follow AdaDexTrack, extended to two hands: hand errors use the worse hand, and a clip succeeds only if both hands meet the thresholds. Best value per column in lime.

Team

Citation