[New CES2021] Discover the teleoperation app

Hi!

We are really proud to finally share an application we are working on for a while (especially @Gaelle and @pierre): teleoperating remotely Reachy using VR!

It’s actualy the first humanoid VR teleoperation app available on the market. You can discover it in this video:

This app allows you to place yourself in the body of a Reachy, in VR, wherever you are in the world, to remotely operate it and carry out complex tasks. With this new functionality, Reachy is able to learn from the demonstration of the humans who control it, which makes application development even easier.

We had a lot of doubts about the user experience for such application but we find out it works very well.
We are really excited to see how this new dev tool will be used to accelerate the development of more meaningful applications.

Why it is a key application for emergence of interactive robots ?

  • You can prototype and conduct user testing of any kind of service offer or assistance without actually have to spend months coding. Just take the control of Reachy and do it.
  • Provide dataset by demonstration to Reachy so it can learn how he should do things in this context
  • Build your autonomous application, deploy Reachy in the real world and ensure continuous reliability by taking the control when Reachy does not how to react.

How it works ?

  • Reachy is connected through the network (local or internet) to the user computer
  • Reachy’s stereo cameras are displayed in the user #VR headset
  • Thanks to its unique Orbita neck joint, Reachy can reproduce naturally the user head motion, avoiding any sickness feeling.
  • #VR controllers 3D position are directly mapped to Reachy hand position, the controller trigger close the gripper
  • haptics vibration give a force feedback

Is it open source ? YES!

7 Likes

A post was merged into an existing topic: Generating Interest & Awareness by letting people try the tech?

This looks great. How do I connect Reachy’s stereo cameras to the VR headset? Is there links to apps to download on both computers?

Please join this topic to follow ongoing public testing

2 Likes

Can a virtual reachy be controlled over steam with oculus quest? I’d like like to perform some tests in a simulation of my workplace for real world applications

1 Like