Dynamixel Issue

Problem summary

One Dynamixel motor in the arm chain does not appear when instantiating the arm in PYLUOS. PyLuos only returns “void_dxl” when just this motor is connected. This motor was previously configured successfully, but disappeared for unknown reasons. All other motors in the chain still identify and can be controlled successfully.

Where is the problem seen?

  • Physically connect the PyLuos modules (dxl and arm USB gate)
  • Follow the procedure for setting up the Luos with Dynamixel
  • Dynamixel is not discovered in the network

Luos setup procedure

Luos Boards are all flashed with the firmware that came with the Reachy Operating system and according to the instructions here: https://pollen-robotics.github.io/reachy-docs/docs/appendices/flash-luos-module/

Our test setup consists of one USB Gate Luos Board and a Dynamixel Luos Board. Power is provided from a 12V power supply and is connected with the Jack Power Input Luos Board. We are measuring the proper voltage levels at all boards.

We are also following the plug-in sequence as detailed here: https://docs.luos.io/pages/demo_boards/boards_list/dxl.html

Run in shell:

pyluos-usb-gate discover

to figure out which USB port the gate is connected to

Launch Python3

python3

Import Device from pylous package and create an object “device” with the correct USB port

from pyluos import Device
device = Device('/dev/ttyUSB1')

Print a list of the modules using:

device.modules

With a single dynamixel connected, the expected output should be as below. We should see that we can detect one dynamixel in the network with ID 1 and alias “dxl_1”

The problem:

  • When we print device.modules, the expected value in the Alias column should be “dxl_1”, but instead it shows: “void_dxl”. This basically means PyLuos cannot see the motor.
  • We can successfully communicate with the ‘problem’ Dynamixel via dynamixels Wizard2.0 software, and connecting via the U2D2 dynamixel dongle. The motor is controllable.
  • All other motors work fine with both PyLuos and the Dynamixel wizard.

When did it start to fail?

It was working at first, and then we tried to connect to the motor via the u2d2 board and it didn’t connect with one computer. This is the only computer that doesn’t connect to the u2d2. In the middle of troubleshooting this problem, we tried probing the leads to verify that power was being delivered. While probing, there was a small short that occurred. However, using another computer to connect with the Dynamixel Wizard and u2d2 (which connected successfully), we could verify that all of the motors were present with their original configurations (baud rate, angle limits, etc.) After connecting with pyluos again, the one dynamixel in question is no longer visible. However, the other dynamixels are visible.

Troubleshooting attempts

None of these made any difference.

  1. Swapped to a different power supply.

  2. Swapped out different dynamixel wires

  3. Re-flashed the Dynamixel luos board and the corresponding USB gate luos board

  4. Performed a factory reset of the motor

  5. Attempted dynamixel detection using : robot.void_dxl.dxl_detect()

    Link: https://community.luos.io/t/i-cant-connect-to-my-dynamixel-servo-void-dxl/203

  6. When using the dxl-util, the baud rate change (1,000,000 bps) is sent successfully to the dynamixel, but the util doesn’t receive a response. We can verify the changed baud rate with the wizard. Subsequent attempts by the util to communicate fail and the motor is not found.

Questions

  • Is there some sort of setting on that dynamixel that needs to be modified or reset in order for it to get back to a known state?
  • What information is the dxl-util writing to the dynamixels?

Hi Dan,

If you can still communicate with the motor with Dynamixel wizard, the motor should be ok :crossed_fingers:

When you perform a factory reset, you should make sure you set the protocol version 1. The version 2 is not supported by pyluos. You can also set the baud rate to 1M to simplify the first connection with pyluos. If you can post a screenshot of all registers value after the reset I’ll have a look. But I think all others values should not be a problem to communicate with the motor.

It basically setting the baud rate to 1M if it’s not already set. Then, change the id and the angle limits.

You can find the source here for more details: https://github.com/pollen-robotics/reachy/blob/master/software/reachy/utils/dxl_config.py

We tried a factory reset. it didn’t seem to make any difference. We purchased a new motor and it worked fine.

I do have suspicions that the factory reset procedure may not be fully working as the baud rate never goes back to factory setting.

There seems to be quite a few issues we haven’t put our fingers on the cause yet.

I was running the Right Arm test script and I could not run the Trajectory Recorder part more than once without one of the Dynamixels freezing in stiff mode and becoming unresponsive.

Unrepeatable or inconsistent errors:

  1. The entire arm freezes in a pose and becomes unresponsive
  2. There is loss of connection with one motor (not consistently any one motor).
  3. One motor doesn’t come back into compliance mode (not consistently any one motor).

Consistant error:
Reachy motors need constant power cycling of depowering Alim board unplugging gate (to remove voltage from USB) repowering Alim board and plugging USB back in).

  • Otherwise one or motors are not detected by pyluos
  • Even with power cycling motors are frequently not detected usually motor 10

Every time code is run correctly, repeating the same piece of code later won’t work. In preparing or a Live Demo last night I power cycled everything. Ran the test script up to the Trajectory recorder to be ready to run it (which consistently worked as prep before) and the delay in running that program let a loss of connection with a motor happen again.

Has anyone come across this before or are most Reachy’s stable enough to not need constant power cycling? Any idea where to start our troubleshooting?

From what you describe, it seems your Dynamixel communication is extremely unreliable. It’s probably causing all the issues you are seeing (pyluos not detecting the correct motors, communication freezing, etc).

When you see such poor communication with the motors it’s usually comes from a broken wire. Unfortunately this can be hard to find and debug.
Indeed, the wire is usually only broken inside and it’s not visible. It usually comes from an overheating (maybe when you add your issue with your broken motor in the first place) and the wire become slightly more rigid.

As all motors are daisy chained any wire could be the issue. We don’t have any better solution than just check them all and try replacing them. Sorry for the bad news :sweat_smile:

We actually used the U2D2 and Dynamixel to test communication upstream and downstream of the daisy chain. This actually identified one of the motors having a communication issue from one side.

The fix was facepalm inducing, we disconnected and reconnected the wire to the elbow and everything worked more reliably.

2 Likes

Now we are working on the left arm! We are running back into the same problem with the same motor. It is completely functional in the Dynamixel wizard but keeps being found as void_dxl.

We resolved the communication issue with the other motors on your suggestion of replacing some wires.

The MX64 shoulder_roll though is still coming back void. We’ve run the device.void_dxl.dxl_detect(), factory reset several times, followed the advice in this thread. We see it and all the other motors single and in daisy chained in the Dynamixel wizard. No matter what we do we still get a void_dxl for that motor. Last session I got tired of debugging it and just bought a new one for the right arm. All the other motors have been fine.

I’d like to try to get ROBOTIS to take the motor back but all the diagnostics from the Dynamixel wizard come back clean.

Any ideas? Has this happened to anyone before?