Reachy2 hal-4 error - frequent system crashes

Users Support
1

Hi,

while teleoperating Reachy2 for a couple hours and lifting 1-2 kg objects, I get frequent disconnections. The robot stays stiff at the current position, disconnects from VR-App and at that moment I cannot even connect to Reachy via dashboard or ssh anymore. I need to do a hard reboot to continue. (does this mean ubuntu crashes? I am not sure)

I was not able to catch the entire Logs that led to this error, because the system crashes. I managed to catch this hal-4 error though that was logged during the teleoperation today:

core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.057544492185115814
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.08640784025192261
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.08595133572816849
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.06384427845478058
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.06384427845478058
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.04463229328393936
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.08127295970916748
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.2624625861644745
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.2756158113479614
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 3 | commands sent 502.16 cmd/s, commands dropped 1.50 req/s, command time: 1.99 (max 27.71) ms
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 3 | states sent 499.99707 req/s
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.28255876898765564
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.3164033591747284
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.34595993161201477
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.34595993161201477
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.34595993161201477
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.37292245030403137
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.37548622488975525
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.2870517373085022
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 2 | states sent 499.99774 req/s
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 2 | commands sent 484.83 cmd/s, commands dropped 0.83 req/s, command time: 2.06 (max 26.53) ms
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.23114489018917084
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.03899966552853584
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.03899966552853584
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 1 | commands sent 484.84 cmd/s, commands dropped 1.60 req/s, command time: 2.06 (max 28.28) ms
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 1 | states sent 500.0093 req/s
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 5 | states sent 499.99637 req/s
core  | [ethercat_master_server-1] [2025-11-24T12:17:17Z INFO  server] GRPC EtherCAT Slave 5 | commands sent 508.75 cmd/s, commands dropped 1.17 req/s, command time: 1.97 (max 26.70) ms
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : 0.0
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.04626099392771721
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.23349858820438385
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.45542803406715393
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.4631185829639435
core  | [ethercat_master_server-1] [2025-11-24T12:17:16Z INFO  server] GRPC EtherCAT Slave 4 | commands sent 484.71 cmd/s, commands dropped 1.40 req/s, command time: 2.06 (max 28.75) ms
core  | [ethercat_master_server-1] [2025-11-24T12:17:16Z INFO  server] GRPC EtherCAT Slave 4 | states sent 500.00177 req/s
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.4631185829639435
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.4631185829639435
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.4625388979911804
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.4602959454059601
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.45696917176246643
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.45696917176246643
core  | [hal-4] [ERROR]: self.x_vel_goal : -0.45666390657424927

I will try to deliver better logs tomorrow. Thank you for any help.

Alexandros

2

Hi @alexandros

Sorry for the log spam, self.x_vel_goal can be discarded and is not the core issue here.
If we suspect a bedrock (the embedded pc) freeze the best way to investigate that is to connect the debug console cable to the computer and see what it is actually up to.
In case of another freeze like this, please plug the cable that should have been shipped with the robot (mini usb) then on a linux computer you can open the console with something like tio /dev/ttyUSB0 (you may need to adapt the USB#)

You will then see a login prompt, or press ENTER if you dont see nothing, and you may then login into reachy2’s computer.
If it does work, then the pc does not freeze, then we may be interested in getting some logs,
journalctl -fu reachy2-core for starters, maybe journalctl -fu webrtc too.
Also dmesg can have some insights.

Please let me know if anything happens again and you can collect some logs !

Clément