Hi everyone, I was trying to start the simulator created by you, Pollen Robotics, within Unity, with the studio scene where there is the desk with the cup, the basketball etc. on it.
To do this, I downloaded the project to open within Unity and it corresponds to the download of the following GitHub repository: GitHub - pollen-robotics/reachy2021-unity-package: Unity package for Reachy 2021
and after that I followed the guide described in the following article: Controlling a Reachy robot in Unity | by Gaelle Lannuzel | pollen-robotics,
which had to lead precisely to my purpose as shown in the image at the end of the article.
Starting the project within Unity, it launches fine.
Initially only the walls of the room are shown, all furniture/objects etc. are missing, including Reachy himself.
It appeared that the objects had been created, but were not visible, as they appeared in the sidebar next to the scene.
By clicking on the Reachy, desk etc. object in the sidebar, the scene highlighted the outlines of the objects making it clear that the objects were present.
Looking at the console, there were many errors describing the lack of all the png files etc. that found the texture of the objects, in reality the files were there, but they all weigh about 1Kb, therefore completely empty.
I leave attached the screenshots of what I described above.
Could you tell me how I could solve and make the whole study scene with the attached furniture appear correctly?
Thanks in advance.
Hello @stefano_caramagno
Somehow, the files seem to have been corrupted, I don’t know how or why.
If you re clone the repo, do you have the same issue ? This should not happen.
As per our previous exchanges, you have been able to get this project working before, what changed ? Is this a different machine ? If so, do you have git lfs installed ?
1 Like
Hi @apirrone,
I tried to clone the repository, now it works, it opens the study scene on Unity correctly.
Regarding the exchange we had weeks ago, no, I couldn’t do it, because as you told me, using the simulator it is impossible to find the z coordinates (depth) with the code.
I had to resize the initial idea of ​​my project and now I’m proceeding like this:
- create codes to make Reachy perform actions, manually like the one at the following link: reachy2021-unity-package-basketball.py · GitHub, therefore knowing the positions of the objects, as available from the Unità simulator;
- train an LLM by providing detailed information on my scripts, images etc.;
- arrange for me to generate code upon my request to make Reachy perform tasks;
- evaluate the efficiency of these codes.
However, I am stuck at the first point, because if I go to compare the file at the following link: reachy2021-unity-package-basketball.py · GitHub, in particular the position of the basketball marked in this code does not correspond to the position of the basketball indicated by Unity by clicking on it.
I leave attached the photo that describes this difference in values.
Can you tell me what this difference is due to and how to find the values ​​present in the code reachy2021-unity-package-basketball.py · GitHub starting from the values ​​provided by Unity by clicking on the basketball?
Thanks in advance.
Hello @stefano_caramagno
As written just above the ball position array in the code, this position is expressed in Reachy’s coordinate system. When you click on the ball in Unity, its position is expressed relative to its parent, here “OfficeScene”. You have to compute a transformation in order to get the ball’s position in Reachy’s coordinate system so that you can use the SDK, in which everything is relative to the robot’s origin.
To do that, there are two ways.
You can find the pose (4x4 homogeneous matrix containing rotation and position) of Reachy in OfficeScene, let’s call it T_office_reachy (reachy expressed in the office’s origin), and the pose of the ball in office, let’s call it T_office_ball (ball expressed in the office’s origin). These two poses are actually what you circled in your image, as both the ball and the robot have the office as their first parent. You have to convert the position and rotation elements to a 4x4 homogeneous matrix.
You want T_reachy_ball (the ball expressed in reachy’s origin), so in python you can do :
T_reachy_office = np.linalg.inv(T_office_reachy)
T_reachy_ball = T_reachy_office @ T_office_ball
Another way to do that is to use Unity’s C# api directly. You should be able to get any object expressed in any other’s coordinate system. Take a look at Unity - Scripting API: Transform and do some googling, this should not be too hard.
Antoine
