Question about the frame in which RDT2 FM model give output in

[OmMandhane]

I was confused about which frame does the model gives output in, and it gives direct coordinates or relative delta. Is the output wrt vive tracker frame, camera frame or base frame of robot.

[LBG21]

Hi, the model output in Vive Tracker frame, and we use convert_policy_to_tcp_space in deploy/umi/real_world/real_inference_util.py to convert the tracker frame to the robot frame. The conversion matrix should first be calibrated via our calibration process. The model gives relative delta action, the same as UMI does.