Reachy 2021 error accessing the cameras

[Alex]

Hi, I’m currently using a Reachy 2021 and I have trouble accessing both cameras.

I tried using the Reachy SDK python API but all I get is this error :

Exception in thread Thread-165: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/home/reachy/reachy-sdk/reachy_sdk/camera.py", line 191, in poll_img for resp in self._stub.StreamImage(stream_request): File "/home/reachy/.local/lib/python3.8/site-packages/grpc/_channel.py", line 426, in __next__ return self._next() File "/home/reachy/.local/lib/python3.8/site-packages/grpc/_channel.py", line 826, in _next raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses" debug_error_string = "{"created":"@1717683654.457389558","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3008,"referenced_errors":[{"created":"@1717683654.457388289","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"

Reachy-sdk is running and everything else works correctly. I’ve checked the script camera.py to see what was the cause of the problem but I could not find anything.

I also tried using the example code view_cam.py with both ROS and OpenCV.
With ROS, I launched camera_publisher.launch.py beforehand and got the following error and so nothing is published on the topic and view_cam.py does nothing.

[INFO] [launch]: All log files can be found below /home/reachy/.ros/log/2024-06-06-15-31-43-158018-reachy-17567 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [camera_publisher-1]: process started with pid [17569] [camera_publisher-1] Traceback (most recent call last): [camera_publisher-1] File "/home/reachy/reachy_ws/install/reachy_controllers/lib/reachy_controllers/camera_publisher", line 33, in <module> [camera_publisher-1] sys.exit(load_entry_point('reachy-controllers==1.0.0', 'console_scripts', 'camera_publisher')()) [camera_publisher-1] File "/home/reachy/reachy_ws/install/reachy_controllers/lib/python3.8/site-packages/reachy_controllers/camera_publisher.py", line 105, in main [camera_publisher-1] camera_publisher = CameraPublisher() [camera_publisher-1] File "/home/reachy/reachy_ws/install/reachy_controllers/lib/python3.8/site-packages/reachy_controllers/camera_publisher.py", line 37, in __init__ [camera_publisher-1] self.left_device.video_capture.set_format(width=resolution[0], height=resolution[1], pixel_format='MJPG') [camera_publisher-1] AttributeError: 'Device' object has no attribute 'video_capture' [ERROR] [camera_publisher-1]: process has died [pid 17569, exit code 1, cmd '/home/reachy/reachy_ws/install/reachy_controllers/lib/reachy_controllers/camera_publisher --ros-args'].

With OpenCV, I managed to access the cameras but the focus is not done.

I tried modifying the code to add focus or autofocus properties but it doesn’t seem to have any effect with these cameras.

   def __init__(self, side: str) -> None:
       """Initialize cap by connecting to the required video port and configuring it."""
       self.cap = cv.VideoCapture(self.side_to_port[side], apiPreference=cv.CAP_V4L2)
       self.cap.set(cv.CAP_PROP_FOURCC, cv.VideoWriter.fourcc('m', 'j', 'p', 'g'))
       self.cap.set(cv.CAP_PROP_FOURCC, cv.VideoWriter.fourcc('M', 'J', 'P', 'G'))
       self.cap.set(cv.CAP_PROP_FPS, 30)
       self.cap.set(cv.CAP_PROP_FRAME_WIDTH, 640)
       self.cap.set(cv.CAP_PROP_FRAME_HEIGHT, 480)
      
       #self.cap.set(cv.CAP_PROP_FOCUS,30)

       # Set focus to auto
       # self.cap.set(cv.CAP_PROP_AUTOFOCUS, 1)  # Turn on autofocus

       self.cam_img = None
       self.rot_value = self.rot[side]

Does anyone know how I can fix this ? (The best way is to use the SDK API, but I can work only with OpenCv if the camera focus is done)
Thanks !

[AnnaelleS]

Hi @Alex !

Based on your logs, I suggest that the issue might be with the version of your v4l2py library. Please run pip show to verify that you have the correct version, as shown below:

Regarding the focus adjustment, follow this procedure:

Make the focus on the right camera :

You will need to open the view_cam.py script AND an ipython terminal side by side to do this.

Open the view_cam window on one side of the screen :

cd ~/reachy_ws/src/reachy_2023/camera_controllers/examples
python3 view_cam.py right ros

In a Python terminal, first perform the zoom homing (it resets the zero positions of the zoom motor for the camera you want to focus):

from reachy_sdk import ReachySDK

reachy = ReachySDK(host='localhost')
reachy.right_camera.zoom_homing()

Change the focus by hand until the image seems clear enough. The value of the focus should be between 100 and 600. For example, to put 300 value for the focus:

reachy.right_camera.focus = 300 

When the image reaches a satisfactory definition, all that’s left to do is close these terminals and do the same for the left camera if necessary, replacing ‘right’ with ‘left’ in the procedure.

[Alex]

Hi, I managed to download the correct v4l2py version and now camera_publisher.launch.py runs without errors and I can now access the cameras with ros.
However the API still produces the same error (only for the cameras, everything else works fine). I tried making the focus and got this :

I think it comes from the gRPC server but I have no knowledge about that.
Thanks !

[AnnaelleS]

Hi,

Thank you for the update. I’m glad to hear that you were able to download the correct v4l2py version and get camera_publisher.launch.py running without errors.

Regarding the issue with the API still producing the same error for the cameras, I have a few questions and suggestions to narrow down the problem:

1. Launch Configuration: Can you confirm that camera_publisher.launch.py is not being launched directly? Instead, please ensure that all services are being launched using run_everything.launch.py. This will help us verify if the error is due to the launch configuration.
2. Configuration Check: Please make sure that nothing is commented out in run_everything.launch.py, as it might have been modified for a special Reachy configuration. This could potentially affect the camera services.

To assist further, could you please provide the reachy.yaml file for review? You can do this by running the following command and sending a screenshot:

nano ~/.reachy.yaml

This will help us verify the configuration settings and identify any possible discrepancies.

Thank you for your cooperation, and I look forward to your response.

Best regards,

Annaëlle

[Alex]

HI, the node camera_publisher is launched with every other nodes at robot startup, and nothing is commented in run_everything.launch.py.

Here is the reachy.yaml file :

Best Regards,

[AnnaelleS]

Hi!

Could you please run a ros2 service list once everything is up and running? This will help us verify that the ROS services for the cameras are starting correctly.

Please send us the resulting list.

Thank you in advance,

Annaëlle.

[Alex]

Hi, here’s the list.

Regards

[AnnaelleS]

Hi !
Open the camera’s video stream with OpenCV and then in another terminal do :

ros2 service call /get_camera_zoom_focus reachy_msgs/srv/GetCameraZoomFocus {}

then

ros2 service call /set_camera_zoom_focus reachy_msgs/srv/SetCameraZoomFocus "{left_focus: {value: 650, flag: true}}"

where you set the value left_focus between 300 and 1000 and see if this has an impact on the focus of your camera, as detailed here :

If the change of focus work with this method, we will more easily identify where the problem lies on the SDK side.

Looking forward to your feedback,
Best regards,

Annaëlle

[Alex]

Hi, I tried changing the focus with this method and it did not work.
Both services gave me the following message :

waiting for service to become available...

I ran ros2 service list to check and both services appear

Looking forward to your answer,

[AnnaelleS]

Hi! Sorry for the delay in responding. We conducted some internal tests and it seems likely that OpenCV is preventing the ROS services from being called.

Does using ROS to view the video stream work? Try replacing the OpenCV command with ROS in the command you use to open it with OpenCV.

If this doesn’t work, could you please send me a screenshot of the logs that appear?

After that, start the Reachy services and try reopening a Python terminal. This time, please do the following:

and send me a screenshot so we can see if the communication with Reachy is working correctly.

Best regards,
Annaëlle

[Alex]

Hi, here’s the screenshot !

As you can see, opening the video stream with ROS works and the communication with reachy seems to work fine but both services are unavaliable

Regards,
Alexandre

[AnnaelleS]

Hi @Alex !

Could you please try calling the services again as previously instructed, but this time make sure to restart the entire PC first to ensure that no old OpenCV connections are present?

Are the camera services available this time?

Additionally, could you go to the reachy_controllers directory and run a git log? The addition of the bgr argument makes me think that the version you are using is different from the one I am using to assist you. But you confirmed earlier that there were no modifications to the cameras, correct? The git log will help me see which version your robot is on.

Thank you for your cooperation. I hope we can resolve your issue as soon as possible.
Best regards,
Annaëlle

[Alex]

Hi !
I tried calling the services after restarting the PC and they still aren’t available.

I ran git log and the most recent commit is this one :

commit 7a84b19150a4563a1359f39c7f0b536db98ff86f (HEAD -> master, origin/master, origin/HEAD)
Author: simheo <simheo0@gmail.com>
Date:   Thu Jul 21 18:24:40 2022 +0200

    Disable autofocus starting automatically at boot.

And I think that there is no modifications to the cameras.

Best regards,
Alexandre

[AnnaelleS]

Hi Alex,

The individual responsible for this commit is currently on leave and will return next week. Upon their return, I should have more detailed information for you.

In the meantime, here’s what I suggest to address the focus issue:

1. First, revert to the previous commit and check if the zoom functions correctly at startup by running:

   git checkout ead74723228695c1e2d20aa859e09f1781f9e13a
   

2. You can then modify the initial zoom level by changing the default_zoom_level variable here:
   default_zoom_level in reachy_controllers

   The possible values are: ‘in’, ‘out’, ‘inter’.

3. If you need more control, you can also adjust the configurations here:
   zoom_piloting configurations

If these tests are not successful, I will provide you with an alternative solution once my colleague returns. In the meantime, you can switch back to the latest version by running:

git switch -

Looking forward to your feedback.

Best regards,
Annaëlle

[Alex]

HI, I tried reverting to the previous commit and these tests were not successful.

Looking forward for the information from your colleague.
Regards
Alexandre

[AnnaelleS]

Hi Alex,

After verifying with the person responsible for this commit, it appears that the autofocus might not be working. Let’s revert to the previous version. We will perform final checks, and if the issue persists, we will schedule a call to take control of your robot and assist you directly.

Here’s what you need to do:

1. Stop Reachy services:

   systemctl --user stop reachy_sdk_server.service
   

2. Add loggers to the services responsible for zoom to ensure they launch correctly during service initialization. They should be added here and here for the Set camera focus and Get camera focus services.

3. Launch only the camera services:

   ros2 launch camera_controllers camera_controllers.launch.py
   

4. Verify the presence of these three services:

   [camera_zoom_service-7] [INFO] [1720685771.484217488] [camera_zoom_controller_service]: Launching "get_camera_zoom_level" service.
   [camera_zoom_service-7] [INFO] [1720685771.488849291] [camera_zoom_controller_service]: Launching "set_camera_zoom_level" service.
   [camera_zoom_service-7] [INFO] [1720685771.505296447] [camera_zoom_controller_service]: Launching "set_camera_zoom_speed" service.
   

   Plus the services where you added the loggers!

If everything works without issues, you can open a window with your camera in ROS and in another terminal run:

ros2 service call /set_camera_zoom_level reachy_msgs/srv/SetCameraZoomLevel "{name: left_eye, zoom_level: homing}"

Then, you can adjust the zoom values to get the desired focus:

ros2 service call /set_camera_zoom_focus reachy_msgs/srv/SetCameraZoomFocus "{left_focus: {value: 610, flag: true}}"

Please send us as many details as possible. If the issue persists, contact me at support@pollen-robotics.com to schedule a meeting.

Thank you for your patience,

Annaëlle

[Alex]

Hi, I have the pleasure to announce you that the issue is finally fixed !

The error was coming from this line and I had the following error

serial.serialutil.SerialException: [Errno 2] could not open port /dev/kurokesu: [Errno 2] No such file or directory: '/dev/kurokesu'

I tried using the default port (dev/ttyACM0) and everything works fine now.

Thank you for your help and your patience !

[AnnaelleS]

Hi, glad to hear it works!
Thanks for your feedback and thanks to you for your involvement.

Have a great day